Repository navigation
test(pricing): allow a source link on Bedrock GovCloud rows the AWS offer file prices - #41167
devin-ai-integration[bot] wants to merge 1 commit into
Conversation
…ffer file prices Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
|
|
Greptile SummaryThis PR permits GovCloud pricing rows to include source metadata by removing the assertion that previously prohibited it
Confidence Score: 4/5The behavior change is reasonable, but the explicit test-integrity requirement must be satisfied by validating source metadata when present The remaining assertions exclude source metadata, so malformed or unrelated provenance can pass without detection; replacing the prohibition with conditional validation preserves coverage Files Needing Attention: tests/test_litellm/test_bedrock_usgov_pricing.py
|
| Filename | Overview |
|---|---|
| tests/test_litellm/test_bedrock_usgov_pricing.py | Removes the blanket source prohibition but leaves optional GovCloud source metadata unconstrained |
Reviews (1): Last reviewed commit: "test(pricing): allow a source link on Be..." | Re-trigger Greptile
| @@ -125,4 +125,3 @@ def test_usgov_rows_keep_commercial_limits_and_capabilities(model_data, gov_key) | |||
| gov = model_data[gov_key] | |||
| assert _non_pricing_fields(gov) == _non_pricing_fields(model_data[GOV_ROW_SOURCES[gov_key]]) | |||
| assert "search_context_cost_per_query" not in gov | |||
There was a problem hiding this comment.
This deletion leaves GovCloud source URLs unchecked, violating the requirement that existing test changes preserve regression coverage before merging
Rule Used: What: Flag any modifications to existing tests and verify they don't weaken test coverage or mask regressions. Why: Developers may alter tests to make failing code pass rather than fix the actual bug, hiding regressions. Good: ``` // Test updated t... (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
TLDR
Problem this solves:
sourcelink on that gov rowtest_bedrock_usgov_pricing.pyasserted gov rows never carrysource, so the sync PR (chore(prices): sync Azure, Azure AI, Gemini, OpenAI, Bedrock, Together AI, Fireworks and Vertex prices: 278 models, 59 new, 30 deprecated #41154) fails CIHow it solves it:
"source" not in govassertion. Capabilities, limits, and the 1.2x premium checks stayUser Flow
Before: the catalog sync PR cannot merge because a test rejects a sourced GovCloud row
sourcelink onus-gov.anthropic.claude-fable-5-1assert 'source' not in {...}for that rowAfter: the same sync PR passes the GovCloud test
sourcelinktest_usgov_rows_keep_commercial_limits_and_capabilitiespasses for all 19 gov rowsRelevant issues
Unblocks #41154
Affected release
Linear ticket
Pre-Submission checklist
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Screenshots / Proof of Fix
Raw provider evidence. The assertion was written on 2026-09-04 (bef3585) when no public AWS source listed Claude GovCloud rows. The AWS Price List offer file https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonBedrockFoundationModels/current/index.json (publicationDate 2026-09-11T12:44:10Z, snapshot in litellm-providers
src/server/pricing/__tests__/fixtures/bedrock-foundation-models.json) now carries us-gov-east-1 products for "Claude Fable 5.1 (Amazon Bedrock Edition)". Per-1M-token USD rates and the catalog value the sync wrote:Every rate is exactly 1.2x the commercial
anthropic.claude-fable-5-1row, so the existing premium test keeps holding. The gov row in #41154 differs frommainonly by the addedsourcelinkBefore (d2859e1)
uv run pytest tests/test_litellm/test_bedrock_usgov_pricing.pyagainst the chore(prices): sync Azure, Azure AI, Gemini, OpenAI, Bedrock, Together AI, Fireworks and Vertex prices: 278 models, 59 new, 30 deprecated #41154 catalogFAILED ...test_usgov_rows_keep_commercial_limits_and_capabilities[us-gov.anthropic.claude-fable-5-1] - AssertionError: assert 'source' not in {...}After (e9770e9)
uv run pytest tests/test_litellm/test_bedrock_usgov_pricing.py22 passed, and the non-pricing fields of the chore(prices): sync Azure, Azure AI, Gemini, OpenAI, Bedrock, Together AI, Fireworks and Vertex prices: 278 models, 59 new, 30 deprecated #41154 gov row still equal the commercial rowType
✅ Test
Caveats (if any)
Low
sourcelink, the test no longer guards against a stale oneFinal Attestation
Link to Devin session: https://app.devin.ai/sessions/a6532463fd19440d8490b0393c54d776
Open in Devin Desktop: https://app.devin.ai/desktop/session/a6532463fd19440d8490b0393c54d776?variant=devin