feat(code): local pricing overrides as a fallback when genai-prices misses - #5304
Merged
Merged
Conversation
…isses When neither the bundled nor the auto-updated genai-prices catalog covers a model, estimate_cost now consults a local override catalog before returning None: - a maintainer-curated bundled_prices.json shipped as package data (empty provider array for now; entries must link an upstream genai-prices PR/issue and be removed once released) - a user-supplied prices.json under the user config directory (~/.deepagents), winning on conflicting (provider id, model id) Fallback-on-miss only: a successful primary calc_price never touches the overrides, so upstream rates always win and a built-in entry goes inert the day a released genai-prices ships the model. Entries use the raw provider-array schema of upstream's data.json, parsed with the same helper UpdatePrices.fetch uses, and pricing goes through ModelInfo.calc_price so bucket decomposition matches a natively cataloged model. Overrides live outside the snapshot mechanism, so the hourly auto-update swap cannot clobber them. Bad override files log once and are ignored; pricing never interrupts a model turn.
github-actions
Bot
added
dcode
![open-swe[bot]](https://avatars.githubusercontent.com/in/1395372?s=60&v=4){kind=small}
Guard the override return with the same finite/non-negative predicate `estimate_cost` applies on its own path. Rates are unbounded `Decimal`s, so a mistyped one converts to `inf` -- and `inf > 0`, so nothing downstream filters it out of a session total. Distinguish transient read failures from deterministic content errors. Only the latter cache: a momentary `PermissionError`, an exhausted fd table, or the window an editor opens while saving `prices.json` used to latch the empty build for the life of the process, reporting "this user has no overrides" indistinguishably from having no file at all. Report a vanished genai-prices helper at WARNING, once, instead of letting `_override_price`'s blanket handler swallow it to DEBUG on every request. `_providers_from_raw` disappearing now caches empty and says so; `find_provider_by_id` disappearing degrades to inference rather than killing the catalog. Warn on a missing or blank bundled resource too -- absent package data is always a defect, unlike an absent user file. Explain an override miss: one WARNING when a non-empty `prices.json` matched nothing, naming the post-alias `provider_id` the lookup actually searched on. Otherwise a user who hand-writes rates and still sees no cost has nothing to go on. Widen the test-isolation fixture to the whole module. Every test that prices an uncovered model now reaches the override loader, so the class-local redirect left the rest of the file reading the developer's real `~/.deepagents/prices.json` -- green in CI, red for anyone who has written one. Correct the docs against the installed genai-prices: precedence is user file first; there is no lock-free read of `_PRICE_OVERRIDES`; the `all_providers` hop is confined to the override catalog; upstream's litellm prefix split and `provider_api_url` matching are not reproduced; and removal tracks upstream's `data.json` rather than a release, since the updater fetches `refs/heads/main`. Also replace `Any` with `ModelInfo`, hoist the duplicated `model_match` inference and source-label construction, and cover the package-resource read, non-array payloads, blank files, the full-sweep path, and the build's once-per-process serialization.
Make the local pricing override path report what it does instead of degrading quietly, and stop the loader from reading the developer's own `~/.deepagents/prices.json` during unrelated tests. Test isolation: the override-state fixture moves from `test_cost_tracking.py` to the unit-test conftest, beside `_disable_prices_auto_update`. Every test that prices an uncatalogued model now reaches the override loader, so `test_session_stats.py` was reading the real user config directory -- green in CI, red for anyone who has written a `prices.json`. Cross-provider sweeps: a sweep that bills a request under a provider it did not run against now warns once naming both, and the success trail names the provider that supplied the rates. Model ids are commonly shared across providers at very different rates, so this is the one path here that can produce a wrong figure rather than no figure. A claim by id or `model_match` stays final and is now pinned by a test. Read failures: only `_TRANSIENT_READ_ERRNOS` suppresses caching. `PermissionError`, `IsADirectoryError`, and a non-UTF-8 file are deterministic and cache like any other outcome; treating them as transient cost a re-read and full re-validation of both catalogs on every unpriced request, silently. All warnings route through `_report_override_once`, so a deterministic problem on one source cannot spam per-request because the sibling failed transiently. Handler scope: the build moves to `_build_price_overrides`, leaving `_price_overrides` a cache-and-safety-net that never raises and warns once on an unhandled load failure. `_override_price` no longer wraps the load, and its own handler warns per model rather than logging at DEBUG -- an entry that matches but cannot be priced is a fixable mistake in a hand-written file. Miss reports key per `(model, provider)` instead of latching once per process, so a side model cannot spend the report a misconfigured model needed, and they name the resolved path rather than the bare filename. Comments: the pin rationale was wrong in three places. The range spans every `0.1.x` patch, so a private upstream name can move on a lockfile refresh with `pyproject.toml` untouched; the changelog to check is upstream's, not ours. Also corrected the single-caller dependency docstring, the `litellm` overclaim, and the provider-level metadata a same-id merge drops. `pyproject.toml` now names both private symbols at the line someone edits when widening the pin. Docs: `PRICING.md` documents `prices.json` for users -- location, the required-field table, the provider-id trap, and a symptom-to-cause table for the Debug Console messages. `bundled_prices.README.md` gains a worked example, required fields, and the caveat that a stale entry is only inert while the primary lookup succeeds. Tests: 12 new, covering negative and non-UTF-8 inputs, both untested resolution rungs, the mismatch warning firing and not firing, `find_provider_by_id` degradation, unexpected load failure, upstream gaining coverage later, the mid-session-edit restart requirement, and per-model miss keying. The bundled-resource test now parses its payload instead of asserting it empty, and a new test turns the README's `price_comments` policy into a gate.
Documenting pricing overrides in the package README is deferred; the canonical reference remains libs/code/PRICING.md and command help.
Mason Daugherty (mdrxy)
added a commit
that referenced
this pull request
Aug 4, 2026
Baseten model cost estimates now fall back to a built-in pricing catalog instead of showing `$0`. All 12 Baseten Model APIs models are priced at their published input / cached-input / output rates. --- [Baseten Model APIs](https://docs.baseten.co/inference/model-apis/overview#supported-models) shipped models that genai-prices does not yet cover, so `estimate_cost` returned `None` for them and those requests dropped out of the session total. This adds a `baseten` provider block to `bundled_prices.json` (#5304's fallback-on-miss override), priced from the [Baseten pricing page](https://www.baseten.co/pricing). This is a stopgap. Every entry carries `price_comments: "Stopgap pending pydantic/genai-prices#549"` per the `bundled_prices.README.md` policy, enforced by `test_every_bundled_override_entry_is_priced_and_links_upstream`. The upstream addition is open at pydantic/genai-prices#549; once that merges and the hourly auto-update picks it up, these entries go inert automatically (upstream always wins on a primary-catalog hit) and can be removed as housekeeping. Rates per 1M tokens (input / cached input / output): | Model | Slug | Input | Cache | Output | |---|---|---|---|---| | DeepSeek V4 Pro | `deepseek-ai/DeepSeek-V4-Pro` | 1.74 | 0.145 | 3.48 | | DeepSeek V4 Flash 0731 | `deepseek-ai/DeepSeek-V4-Flash-0731` | 0.13 | 0.028 | 0.26 | | GLM 4.7 | `zai-org/GLM-4.7` | 0.60 | 0.12 | 2.20 | | GLM 5.2 | `zai-org/GLM-5.2` | 1.40 | 0.14 | 4.40 | | GLM 5.2 Fast | `zai-org/GLM-5.2-Fast` | 2.10 | 0.21 | 6.60 | | Inkling | `thinkingmachines/inkling` | 1.00 | 0.17 | 4.05 | | Inkling Small | `thinkingmachines/inkling-small` | 0.50 | 0.10 | 1.20 | | Kimi K2.6 | `moonshotai/Kimi-K2.6` | 0.95 | 0.16 | 4.00 | | Kimi K2.7 Code | `moonshotai/Kimi-K2.7-Code` | 0.95 | 0.16 | 4.00 | | Kimi K3 | `moonshotai/Kimi-K3` | 3.00 | 0.30 | 15.00 | | NVIDIA Nemotron 3 Ultra | `nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B` | 0.60 | 0.12 | 2.40 | | OpenAI GPT 120B | `openai/gpt-oss-120b` | 0.10 | — | 0.50 | Notes: - Cached input uses `cache_read_mtok`, matching Baseten's "Cache Input" KV-cache rate (applied automatically to every request). - `openai/gpt-oss-120b` publishes no separate cached-input rate, so `cache_read_mtok` is omitted — cached tokens stay in the ordinary input total. - Extra `match` aliases cover the bare slugs Baseten's pricing "Try" links use: `glm-4-7`, `inkling`, `inkling-small`.
Mason Daugherty (mdrxy)
pushed a commit
that referenced
this pull request
Aug 6, 2026
> [!CAUTION] > Merging this PR will automatically publish to **PyPI** and create a **GitHub release**. For the full release process, see [`.github/RELEASING.md`](https://github.com/langchain-ai/deepagents/blob/main/.github/RELEASING.md). --- _Release notes preview: keep this section in sync with the package `CHANGELOG.md`. Publish reads the merged CHANGELOG via `release.yml`, not this PR description — keep them aligned anyway so the PR stays an accurate historical record for reviewers and anyone returning later._ --- ## [0.1.53](deepagents-code==0.1.52...deepagents-code==0.1.53) (2026-08-06) ### Features - Added pricing coverage with Baseten built-in overrides and local fallback overrides when `genai-prices` is missing data ([#5312](#5312), [#5304](#5304)). - Suggest compacting large resumed threads ([#5318](#5318)). - Added terminal program trace metadata ([#5329](#5329)). ### Bug Fixes - Preserved runtime offload archive routing ([#5328](#5328)). - Always restart after a successful startup auto-update ([#5317](#5317)). - Fixed leaked turn coroutines and SQLite handles ([#5218](#5218)). - Keep MCP shutdown-race tracebacks from appearing in the terminal ([#5325](#5325)). - Open the `/auto model` selector immediately while connecting ([#5341](#5341)). - Route failures to `PostToolUseFailure` ([#5315](#5315)). - Use dismissed copy for ask-user prompts ([#5331](#5331)). _End release notes preview._ --- > [!NOTE] > A **New Contributors** section is appended to the GitHub release notes automatically at publish time (see [Release Pipeline](https://github.com/langchain-ai/deepagents/blob/main/.github/RELEASING.md#release-pipeline), step 2). --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: langchain-oss-automated-triage[bot] <248757908+langchain-oss-automated-triage[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cost estimates now fall back to a local pricing catalog when genai-prices has no rates for a model. Two sources are consulted, in precedence order: a user-supplied
prices.jsonin the user config directory (~/.deepagents) for models upstream lacks entirely, and a maintainer-curatedbundled_prices.jsonshipped inside the package as a stopgap while an upstream addition is pending. The fallback only fires on a primary-catalog miss, so published upstream rates always win.estimate_costpreviously returnedNonefor any model the genai-prices catalog — bundled or hourly auto-updated (#5264) — did not cover, leaving those requests out of the session total with no recourse short of waiting on a genai-prices release. This adds an escape hatch without giving up upstream as the source of truth.Design points:
Fallback-on-miss, upstream always wins. Overrides are consulted only from the existing
except LookupErrorpath inestimate_cost; they are never installed viaset_custom_snapshot. The hourly auto-updater wholesale-replaces the custom snapshot, so anything installed there would be clobbered every refresh — keeping overrides outside the snapshot mechanism is what makes the two features compose. A successful primary lookup never touches the overrides, which means a built-in entry automatically becomes dead weight the day a released genai-prices ships the model, with no migration needed.Worked examples
Same schema as upstream. Both files use the raw provider-array schema of genai-prices'
prices/new_data/v2/data.jsonUser file wins on conflict. On a conflicting
(provider id, model id)pair, the user'sprices.jsonentry replaces the built-in one (per-model). The user path resolves through the existingmodel_config.DEFAULT_CONFIG_DIRconstantNever breaks a model turn. A missing user file is the normal case and stays silent. Malformed JSON, a non-array payload, an unreadable file, or a schema validation failure each log a warning once and disable only that source;
pricing_data_available()and the contract-broken guard are untouched, so a bad override file never masquerades as a broken pricing install. Successful override pricing logs at DEBUG — that's the signal to pursue the upstream addition.Built-in ships empty.
bundled_prices.jsonis a structurally valid empty provider array until the first stopgap is needed. The maintenance policy lives in the siblingbundled_prices.README.md(JSON has no comments): every entry must link the upstream genai-prices PR/issue, and must be removed once a released genai-prices covers it.