[steebchen]

Summary

On AI Studio (Gemini Developer API), service_tier flex/priority does work — a direct call returns the served tier. But on streaming responses Google omits the x-gemini-service-tier response header and instead reports the served tier in the body as usageMetadata.serviceTier (flex / priority / standard). Unary responses include both.

The gateway only read the header, so every streamed Flex/Priority AI Studio request resolved to null (standard) — mis-billing, and making the selected tier look like it did nothing.

resolveServedServiceTier now also reads the usageMetadata.serviceTier body field; the header still takes precedence when present.

Verification

Verified against the live Google API with gemini-3.1-pro-preview:

• Direct AI Studio call → unary response has both header x-gemini-service-tier: flex and body usageMetadata.serviceTier: flex; streaming response has only the body field.
• Through the gateway (streaming): AI Studio flex now resolves flex, priority resolves priority (previously both resolved to standard).
• Vertex is unaffected (served tier still resolved from usageMetadata.trafficType).
• pnpm format, full pnpm build, and unit tests pass (18 tests in apply-google-service-tier.spec.ts, incl. new body-field cases).

Scope

This PR only fixes served-tier detection for AI Studio streaming. It does not change how web search is routed across service tiers (a separate Vertex Flex + grounding change was considered and dropped for now).

🤖 Generated with Claude Code

[coderabbitai]

Walkthrough

Adds usageMetadata.serviceTier as a fallback input for served service-tier resolution. The gateway chat paths now pass that value into resolveServedServiceTier, and the resolver tests cover body-based mapping and fallback behavior.

Changes

Google service-tier body fallback

Layer / File(s)	Summary
Resolver contract and parsing packages/actions/src/apply-google-service-tier.ts	Updates resolveServedServiceTier to accept serviceTierBody and resolve the served tier from the header or body value.
Chat call-site wiring apps/gateway/src/chat/chat.ts	Passes usageMetadata.serviceTier into resolveServedServiceTier from the streaming and non-streaming Google-compatible chat paths.
Resolver tests packages/actions/src/apply-google-service-tier.spec.ts	Adds coverage for serviceTierBody mapping, header-null fallback, and the all-null no-signals case.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• theopenco/llmgateway#2549: Introduced the service-tier resolution path that this PR extends with serviceTierBody.
• theopenco/llmgateway#2819: Touches the same gateway service-tier plumbing and logging path affected by these changes.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title matches a major part of the PR: detecting the served tier from AI Studio streaming responses.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch gemini-flex-web-search-empty

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (1)

packages/actions/src/apply-google-service-tier.spec.ts (1)

97-103: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Assert actual header precedence here.

This only verifies the null-header fallback, so a regression that starts preferring serviceTierBody when both are present would still pass. Add one conflicting-value case in this test.

Suggested assertion

 it("prefers the header but falls back to the body field", () => {
+	expect(
+		resolveServedServiceTier({
+			serviceTierHeader: "priority",
+			serviceTierBody: "flex",
+		}),
+	).toBe("priority");
 	expect(
 		resolveServedServiceTier({
 			serviceTierHeader: null,
 			serviceTierBody: "flex",
 		}),
 	).toBe("flex");
 });

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/actions/src/apply-google-service-tier.spec.ts` around lines 97 -
103, The resolveServedServiceTier test only covers the null-header fallback, so
it misses a regression where serviceTierBody could win when both values exist;
update the existing spec in apply-google-service-tier.spec.ts to add a
conflicting-value case for resolveServedServiceTier that explicitly asserts the
header takes precedence over the body while keeping the null-header fallback
check.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@packages/actions/src/apply-google-service-tier.spec.ts`:
- Around line 97-103: The resolveServedServiceTier test only covers the
null-header fallback, so it misses a regression where serviceTierBody could win
when both values exist; update the existing spec in
apply-google-service-tier.spec.ts to add a conflicting-value case for
resolveServedServiceTier that explicitly asserts the header takes precedence
over the body while keeping the null-header fallback check.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: cfacdba5-36db-4871-b12b-019334468358

📥 Commits

Reviewing files that changed from the base of the PR and between 5be8800 and c5544ca.

📒 Files selected for processing (3)

• apps/gateway/src/chat/chat.ts
• packages/actions/src/apply-google-service-tier.spec.ts
• packages/actions/src/apply-google-service-tier.ts

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c5544ca817

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".