tests: add bedrock_mantle provider capabilities and Postman environment config

[Pratham-Mishra04]

Summary

Adds E2E test configuration and capability definitions for the bedrock_mantle provider, enabling it to be tested through the Bifrost V1 API test suite.

Changes

• Added bedrock_mantle to provider-capabilities.json with chat_completions, chat_completions_with_tools, responses, responses_with_tools, and list_models enabled
• Added a new Postman environment file (bifrost-v1-bedrock-mantle.postman_environment.json) configured to use anthropic.claude-opus-4-8 as the default model and us-east-1 as the default region, with secret placeholders for API key, access key, secret key, and session token

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (React)
• Docs

How to test

Run the E2E test suite targeting the bedrock_mantle provider using the new Postman environment:

# Ensure the Bifrost server is running locally on port 8080
# Load the environment file and run the collection against bedrock_mantle

newman run tests/e2e/api/bifrost-v1.postman_collection.json \
  -e tests/e2e/api/provider_config/bifrost-v1-bedrock-mantle.postman_environment.json \
  --env-var "bedrock_mantle_api_key=<your_api_key>" \
  --env-var "bedrock_mantle_access_key=<your_access_key>" \
  --env-var "bedrock_mantle_secret_key=<your_secret_key>"

Expected outcome: chat completions, tool-use, responses, and model listing tests pass; all unsupported capability tests are skipped or return expected errors.

Screenshots/Recordings

N/A

Breaking changes

• Yes
• No

Related issues

N/A

Security considerations

The Postman environment file stores API key, access key, secret key, and session token as secret type fields with empty default values, ensuring credentials are not committed to the repository.

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[Pratham-Mishra04]

• tests: add bedrock_mantle provider capabilities and Postman environment config #4739 👈 (View in Graphite)
• feat: add AWS Bedrock Mantle provider docs #4738
• feat: add bedrock_mantle provider with SigV4 key config, DB migration, and UI support #4737
• feat: add bedrock_mantle as a first-class provider with native-Anthropic and OpenAI-compatible routing #4736
• refactor: replace pre-built SigV4 body signing with lazy BodySigner closure passed through request handlers #4735
• refactor: introduce BearerAuthHeader helper and migrate OpenAI-compatible providers from schemas.Key to map[string]string auth header param #4425 : 1 other dependent PR (#4709 )
• refactor: extract HandleAnthropicChatCompletionRequest / HandleAnthropicResponsesRequest and make completeRequest a package-level func shared by Anthropic, Azure, and Bedrock providers #4394
• refactor: centralize Anthropic request building into BuildAnthropicChatRequestBody and AnthropicProviderRequestDefaultsMap #3309 : 1 other dependent PR (#4360 )
• dev

This stack of pull requests is managed by Graphite. Learn more about stacking.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Confidence Score: 4/5

Safe to merge for parallel-mode CI runs; sequential-mode monitoring attribution is broken when both bedrock and bedrock_mantle are in the provider list.

The harness-monitor.mjs update only fixes the RE_PREFIX regex for parallel-mode log tagging but does not add bedrock_mantle to PROVIDER_KEYWORDS. In sequential mode, inferProviderFromLine iterates providers in order and matches the bedrock keyword — which is a substring of "bedrock_mantle" — before reaching the bedrock_mantle entry, so bedrock_mantle progress lines are silently claimed by the bedrock partition. The equivalent fix was correctly applied to filter-collection.mjs with both the new keyword and a disambiguation guard, but was not carried through to the monitor. All other changes (capability matrix, Postman environment, provider-harness reformatting, go.mod bumps) look correct.

tests/e2e/api/runners/harness-monitor.mjs — needs bedrock_mantle added to PROVIDER_KEYWORDS and a disambiguation guard matching the one added in filter-collection.mjs.

Important Files Changed

Filename	Overview
tests/e2e/api/runners/harness-monitor.mjs	Fixes RE_PREFIX regex for parallel-mode log tagging but omits bedrock_mantle from PROVIDER_KEYWORDS, causing sequential-mode stats to be misattributed to bedrock.
tests/e2e/api/runners/filter-collection.mjs	Correctly adds bedrock_mantle to PROVIDER_KEYWORDS and adds an explicit disambiguation guard to prevent bedrock from claiming bedrock_mantle rows.
tests/e2e/api/provider-capabilities.json	Adds bedrock_mantle capability matrix; flags match the actual provider implementation (responses/responses_with_tools true, all others false as per bedrockmantle.go).
tests/e2e/api/provider_config/bifrost-v1-bedrock-mantle.postman_environment.json	New Postman environment for bedrock_mantle; credentials are correctly typed as secret with empty defaults; region defaults to us-east-1.
tests/e2e/api/collections/provider-harness.json	Reformats existing variables from compact to pretty-printed JSON; no functional changes to the collection.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[newman run] -->|parallel mode| B["newman-cli-bedrock_mantle.log"]
    A -->|sequential mode| C[newman-cli.log]

    B --> D["harness-monitor.mjs\n RE_PREFIX regex\n now matches [bedrock_mantle]"]
    C --> E["inferProviderFromLine()\n PROVIDER_KEYWORDS lookup"]

    E -->|bedrock_mantle NOT in PROVIDER_KEYWORDS| F["fallback: keywords = ['bedrock_mantle']"]
    E -->|bedrock IS in PROVIDER_KEYWORDS before bedrock_mantle| G["'bedrock' keyword matches\n bedrock_mantle lines first ⚠️"]

    D --> H[Correct per-provider stats]
    F --> I["May work if bedrock not also running"]
    G --> J["bedrock_mantle stats\n attributed to bedrock ❌"]

    K["filter-collection.mjs"] -->|correctly disambiguates| L["bedrock_mantle rows\n excluded from bedrock partition ✓"]

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[newman run] -->|parallel mode| B["newman-cli-bedrock_mantle.log"]
    A -->|sequential mode| C[newman-cli.log]

    B --> D["harness-monitor.mjs\n RE_PREFIX regex\n now matches [bedrock_mantle]"]
    C --> E["inferProviderFromLine()\n PROVIDER_KEYWORDS lookup"]

    E -->|bedrock_mantle NOT in PROVIDER_KEYWORDS| F["fallback: keywords = ['bedrock_mantle']"]
    E -->|bedrock IS in PROVIDER_KEYWORDS before bedrock_mantle| G["'bedrock' keyword matches\n bedrock_mantle lines first ⚠️"]

    D --> H[Correct per-provider stats]
    F --> I["May work if bedrock not also running"]
    G --> J["bedrock_mantle stats\n attributed to bedrock ❌"]

    K["filter-collection.mjs"] -->|correctly disambiguates| L["bedrock_mantle rows\n excluded from bedrock partition ✓"]

Loading

Comments Outside Diff (1)

1. tests/e2e/api/runners/harness-monitor.mjs, line 53-61 (link)

   Sequential-mode bedrock_mantle stats misattributed to bedrock

   The comment on line 51 explicitly says "Mirror filter-collection.mjs PROVIDER_KEYWORDS", but bedrock_mantle was not added here. In sequential mode, inferProviderFromLine iterates PROVIDERS in order and resolves each against PROVIDER_KEYWORDS[p] || [p]. For bedrock, the keyword is ["bedrock"]; since "bedrock" is a substring of "bedrock_mantle", any line containing "bedrock_mantle" is claimed by the bedrock partition before reaching bedrock_mantle. The PR correctly updated filter-collection.mjs with both the new keyword and the explicit disambiguation guard, but those changes were not carried into this file. The RE_PREFIX regex fix ([a-z_]+) only covers parallel-mode log prefixes; sequential-mode line routing is still broken.

_{Reviews (8): Last reviewed commit: "feat: adds harness tests for bedrock man..." | Re-trigger Greptile}

[coderabbitai]

Warning

Review limit reached

@Pratham-Mishra04, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 2 minutes and 46 seconds. Learn how PR review limits work.

To continue reviewing without waiting, enable usage-based billing in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: b7132064-faeb-4ed7-8e37-632cd548d152

📥 Commits

Reviewing files that changed from the base of the PR and between f7426db and 94f2cd5.

📒 Files selected for processing (8)

• tests/cmd/e2eseed/go.mod
• tests/cmd/seed/go.mod
• tests/cmd/seedvks/go.mod
• tests/e2e/api/collections/provider-harness.json
• tests/e2e/api/provider-capabilities.json
• tests/e2e/api/provider_config/bifrost-v1-bedrock-mantle.postman_environment.json
• tests/e2e/api/runners/filter-collection.mjs
• tests/e2e/api/runners/harness-monitor.mjs

📝 Walkthrough

Walkthrough

Adds bedrock_mantle provider fixtures and updates e2e runner parsing and dependency pins for the new provider partition.

Changes

Bedrock Mantle provider support

Layer / File(s)	Summary
Capability matrix and environment tests/e2e/api/provider-capabilities.json, tests/e2e/api/provider_config/bifrost-v1-bedrock-mantle.postman_environment.json	A bedrock_mantle capability entry is added, and a Postman environment defines base URL, provider/model defaults, secret placeholders, and default identifiers.
Runner filtering and module pins tests/e2e/api/runners/filter-collection.mjs, tests/e2e/api/runners/harness-monitor.mjs, tests/cmd/*/go.mod	The e2e filter treats bedrock_mantle as a separate provider partition, the harness parser accepts underscore-tagged provider prefixes, and the seed module dependency versions move to github.com/maximhq/bifrost/core v1.6.0.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Suggested reviewers

• akshaydeo
• danpiths

Poem

🐇 I hopped by Mantle, light and spry,
New flags and fixtures passed me by.
The logs now read my underscore name,
And test-path carrots know my lane.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: adding bedrock_mantle provider capabilities and Postman config for tests.
Description check	✅ Passed	The PR description follows the required template and includes the key sections, testing steps, and security notes.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 06-26-feat_adds_harness_tests_for_bedrock_mantle

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@tests/e2e/api/provider_config/bifrost-v1-bedrock-mantle.postman_environment.json`:
- Around line 1-120: The Newman environment for the Bedrock Mantle provider is
missing shared harness variables that the API suite expects, so unresolved
placeholders can break test execution before provider assertions run. Update the
environment file used by the Postman collection to include the common values
referenced across the suite, especially the shared aliases and IDs such as
baseUrl, admin_username, openaiKey, vertex* fields, and the vk*/logid*
variables, alongside the existing bedrock_mantle-specific settings.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: c17fd436-5402-4944-9178-e59e9d7bd90d

📥 Commits

Reviewing files that changed from the base of the PR and between 53b65c2 and 52f40d0.

📒 Files selected for processing (2)

• tests/e2e/api/provider-capabilities.json
• tests/e2e/api/provider_config/bifrost-v1-bedrock-mantle.postman_environment.json

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/e2e/api/runners/filter-collection.mjs`:
- Around line 113-117: The exclusivity check in the filter logic only recognizes
the underscored Mantle alias, so hyphenated rows can still be captured by the
generic bedrock match. Update the Mantle detection in the relevant filtering
helper around isMantle/PROVIDER handling to treat both “bedrock_mantle” and
“bedrock-mantle” as Mantle before falling through to bedrock, keeping the
provider routing exclusive for both aliases.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 2d5f74cc-e602-4c5b-a638-5bd5db598818

📥 Commits

Reviewing files that changed from the base of the PR and between 52f40d0 and fec160b.

📒 Files selected for processing (8)

• tests/cmd/e2eseed/go.mod
• tests/cmd/seed/go.mod
• tests/cmd/seedvks/go.mod
• tests/e2e/api/collections/provider-harness.json
• tests/e2e/api/provider-capabilities.json
• tests/e2e/api/provider_config/bifrost-v1-bedrock-mantle.postman_environment.json
• tests/e2e/api/runners/filter-collection.mjs
• tests/e2e/api/runners/harness-monitor.mjs

🚧 Files skipped from review as they are similar to previous changes (2)

• tests/e2e/api/provider_config/bifrost-v1-bedrock-mantle.postman_environment.json
• tests/e2e/api/provider-capabilities.json

[akshaydeo]

Merge activity

• Jun 27, 3:42 PM UTC: A user started a stack merge that includes this pull request via Graphite.
• Jun 27, 3:43 PM UTC: Graphite couldn't merge this pull request because a downstack PR refactor: extract HandleAnthropicChatCompletionRequest / HandleAnthropicResponsesRequest and make completeRequest a package-level func shared by Anthropic, Azure, and Bedrock providers #4394 failed to merge.
• Jun 27, 4:50 PM UTC: A user started a stack merge that includes this pull request via Graphite.
• Jun 27, 5:06 PM UTC: Graphite rebased this pull request as part of a merge.
• Jun 27, 5:07 PM UTC: @Pratham-Mishra04 merged this pull request with Graphite.

The base branch was changed.