chore: additional harness tests

[TejasGhatte]

Summary

Adds E2E harness test coverage for two recently merged PRs: Gemini web_search_options → googleSearch tool mapping with url_citation annotation passback (PR #5139), and OpenAI Responses API reasoning.effort normalization for max/xhigh values across GPT-5.x model variants (PR #5130). The backlog item for web_search_options is also marked as complete.

Changes

• Marks web_search_options as covered in HARNESS_COVERAGE_BACKLOG.md
• Adds harness collection group 22 covering Gemini web_search_options → googleSearch translation, asserting that url_citation annotations are present in both non-streaming and SSE streaming responses (previously the option was silently dropped and no annotations were returned)
• Adds harness collection group 23 covering normalizeOpenAIReasoningEffort behavior:
  • gpt-5.6 passes effort: "max" through unchanged and the response echoes it back as "max"
  • gpt-5 downgrades effort: "max" to "high" so the upstream does not return a 400
  • gpt-5.6 accepts and echoes reasoning.context and reasoning.mode fields that were previously stripped before serialization

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (React)
• Docs

How to test

Import tests/e2e/api/collections/provider-harness.json into Postman or Newman and run groups 22 and 23 against a live environment with valid API keys for Gemini and OpenAI.

newman run tests/e2e/api/collections/provider-harness.json \
  --env-var baseUrl=<your-proxy-url> \
  --folder "22. Gemini web_search_options → googleSearch (PR #5139)" \
  --folder "23. OpenAI Responses reasoning.effort max normalization (PR #5130)"

Each test skips gracefully on 401/403/429/5xx so transient upstream errors do not cause false failures. A passing run confirms annotations are present for Gemini grounding responses and that reasoning.effort values are correctly normalized or passed through depending on the model.

Screenshots/Recordings

N/A

Breaking changes

• No

Related issues

Closes coverage tracking for PR #5139 and PR #5130.

Security considerations

None. Tests only assert on response shape; no credentials are stored in the collection.

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[coderabbitai]

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 52 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 2d56b7df-9aad-4708-8170-2aec3e887de8

📥 Commits

Reviewing files that changed from the base of the PR and between c143166 and 4bf1bf6.

📒 Files selected for processing (2)

• tests/e2e/api/HARNESS_COVERAGE_BACKLOG.md
• tests/e2e/api/collections/provider-harness.json

📝 Walkthrough

Walkthrough

The provider harness adds Gemini web_search_options coverage for streaming and non-streaming chat completions, and OpenAI Responses coverage for reasoning effort normalization and serialization of context and mode.

Changes

Provider harness coverage

Layer / File(s)	Summary
Gemini web search annotations tests/e2e/api/collections/provider-harness.json, tests/e2e/api/HARNESS_COVERAGE_BACKLOG.md	Adds streaming and non-streaming Gemini chat-completions assertions for url_citation annotations and marks web_search_options as covered.
OpenAI reasoning normalization tests/e2e/api/collections/provider-harness.json	Adds Responses assertions for reasoning.effort: "max", gpt-5 normalization to "high", and reasoning.context/mode echoing.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• maximhq/bifrost#5130: Covers the same OpenAI reasoning effort normalization and serialization behavior.
• maximhq/bifrost#5139: Covers the Gemini web search mapping and url_citation streaming behavior.

Suggested reviewers: akshaydeo, danpiths, pratham-mishra04, roroghost17

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Title check	❓ Inconclusive	The title is relevant but too generic to describe the main harness coverage changes.	Use a more specific title like "chore: add Gemini and OpenAI Responses harness coverage".

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description check	✅ Passed	The description follows the template well and includes summary, changes, testing, and risk sections.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 07-13-chore_additional_harness_tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[TejasGhatte]

• chore: additional harness tests #5155 👈 (View in Graphite)
• chore: test fixes #5152
• dev

This stack of pull requests is managed by Graphite. Learn more about stacking.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Confidence Score: 5/5

This looks safe to merge.

• No blocking issues found in the changed code.

Important Files Changed

Filename	Overview
tests/e2e/api/HARNESS_COVERAGE_BACKLOG.md	Updates the backlog checkbox for Chat Completions web_search_options harness coverage.
tests/e2e/api/collections/provider-harness.json	Adds provider harness groups for Gemini web search annotations and OpenAI Responses reasoning behavior.

_{Reviews (2): Last reviewed commit: "chore: additional harness tests" | Re-trigger Greptile}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

tests/e2e/api/collections/provider-harness.json (1)

37671-37674: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Coverage gap: gpt-5.2/gpt-5.5 → xhigh normalization branch is untested.

The section's own description states the normalization has three outcomes: gpt-5.6 passes max/xhigh through unchanged, gpt-5.2/5.5 downgrade max to xhigh, and other models downgrade to high. Only the first and third branches have test cases (lines 37675-37720 and 37721-37766); the gpt-5.2/5.5 → xhigh branch has no coverage at all, so a regression in that specific mapping would go undetected by this harness.

Do you want me to add a test case for this branch? A minimal addition following the existing pattern:

{
  "name": "openai/gpt-5.2 native /v1/responses reasoning.effort=max downgraded to xhigh - PR `#5130`",
  "request": {
    "method": "POST",
    "header": [{ "key": "Content-Type", "value": "application/json" }],
    "body": {
      "mode": "raw",
      "raw": "{\n  \"model\": \"openai/gpt-5.2\",\n  \"input\": \"Reply with the single word: ok\",\n  \"reasoning\": {\n    \"effort\": \"max\"\n  },\n  \"max_output_tokens\": 512\n}"
    },
    "url": { "raw": "{{baseUrl}}/v1/responses", "host": ["{{baseUrl}}"], "path": ["v1", "responses"] }
  },
  "event": [
    {
      "listen": "test",
      "script": {
        "type": "text/javascript",
        "exec": [
          "if ([401, 403, 429, 500, 502, 503, 504].indexOf(pm.response.code) !== -1) { return; }",
          "pm.test('gpt-5.2 effort max: 2xx', function () { pm.expect(pm.response.code, 'failed: ' + pm.response.text()).to.be.below(400); });",
          "if (pm.response.code >= 400) { return; }",
          "var r = pm.response.json();",
          "pm.test('gpt-5.2 effort max: normalized to xhigh', function () { pm.expect(r.reasoning, 'no reasoning echo').to.be.an('object'); pm.expect(r.reasoning.effort).to.eql('xhigh'); });"
        ]
      }
    }
  ]
}

Also applies to: 37721-37766

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/e2e/api/collections/provider-harness.json` around lines 37671 - 37674,
Add a test item to the existing “OpenAI Responses reasoning.effort max
normalization” section covering model openai/gpt-5.2 with reasoning.effort set
to max. Follow the neighboring response tests’ request and skip handling, then
assert successful responses echo reasoning.effort as xhigh; preserve the
existing gpt-5.6 and other-model cases.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/e2e/api/collections/provider-harness.json`:
- Around line 37767-37812: Update the test script for “gpt-5.6
reasoning.context/mode” so it requires r.reasoning to be present and directly
asserts reasoning.context equals “current_turn” and reasoning.mode equals
“standard”; remove the conditional field guards while preserving the existing
response-status handling.

---

Nitpick comments:
In `@tests/e2e/api/collections/provider-harness.json`:
- Around line 37671-37674: Add a test item to the existing “OpenAI Responses
reasoning.effort max normalization” section covering model openai/gpt-5.2 with
reasoning.effort set to max. Follow the neighboring response tests’ request and
skip handling, then assert successful responses echo reasoning.effort as xhigh;
preserve the existing gpt-5.6 and other-model cases.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 41f991b3-2251-44b5-8534-866e8139bfb7

📥 Commits

Reviewing files that changed from the base of the PR and between ccee45b and 60a22fd.

📒 Files selected for processing (2)

• tests/e2e/api/HARNESS_COVERAGE_BACKLOG.md
• tests/e2e/api/collections/provider-harness.json

[Pratham-Mishra04]

Merge activity

• Jul 13, 2:17 PM UTC: A user started a stack merge that includes this pull request via Graphite.
• Jul 13, 2:21 PM UTC: Graphite rebased this pull request as part of a merge.
• Jul 13, 2:23 PM UTC: Graphite rebased this pull request as part of a merge.
• Jul 13, 2:24 PM UTC: @Pratham-Mishra04 merged this pull request with Graphite.

The base branch was changed.