Repository navigation
Conversation
|
| }, | ||
| "bing_grounding/search": { | ||
| "input_cost_per_query": 0.035, | ||
| "input_cost_per_query": 0.014, |
There was a problem hiding this comment.
Updated test_connection_mode_tracks_the_g1_cost to assert 0.014.
| def test_bing_grounding_pricing(): | ||
| model_info = _load(MAIN_PATH)[MODEL] | ||
|
|
||
| assert model_info["input_cost_per_query"] == BING_GROUNDING_COST_PER_QUERY |
There was a problem hiding this comment.
Price test misses charged result
These tests read JSON but never check what a search charges. They also pin a vendor price without a source and date. The repository requires behavioral regression tests and dated provenance for external facts; satisfy that requirement before merging
Context Used: AGENTS.md (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
There was a problem hiding this comment.
Removed the JSON-only test. Coverage is now the behavioral test_connection_mode_tracks_the_g1_cost, which checks the charged response_cost and cites the dated pricing source. CI's diff check already keeps the main and backup JSON in sync.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
6b3a4de to
70addc0
Compare
|
Superseded by the rolling registry PR #42951, which carries these values re-verified against the official sources with a co-author credit. Thanks! |
TLDR
Problem this solves:
How it solves it:
bing_grounding/searchpricing to $14 per 1,000 ($0.014 per query).diffcheck).test_connection_mode_tracks_the_g1_cost, which runslitellm.searchand asserts the chargedresponse_costis0.014, with a dated pricing source.Official Microsoft pricing (accessed 2026-09-24):
https://www.microsoft.com/en-us/bing/apis/grounding-pricing
User Flow
Before: a user running Bing Grounding searches sees inflated spend.
After: the same usage uses Microsoft's current pricing.
Web-search mode (no connection ID) still charges $0, covered by
test_web_search_mode_is_not_billed_the_g1_price.Pre-Submission checklist
Screenshots / Proof of Fix
Before (57eb3ff)
bing_grounding/searchprice from the cost map.After (35dcc33)
bing_grounding/searchprice from the updated cost map.Tests:
Type
🐛 Bug Fix
Final Attestation