fix(logging): resolve model_map_value for proxy custom pricing by Sameerlite · Pull Request #31940 · BerriAI/litellm

Sameerlite · 2026-07-02T05:39:47Z

Fixes LIT-3719

Summary

Fix model_map_value being null in StandardLoggingPayload when proxy model-group aliases override response.model and custom pricing is enabled
Fall back to metadata["deployment"] for cost-map lookup when base_model is unset
Flush stdout when printing standard logging payloads to avoid delayed output under piped logging

Note

Low Risk
Changes only how standard logging resolves cost-map metadata for proxy/router aliases; request handling and billing math elsewhere are untouched, with targeted tests.

Overview
Fixes null model_map_value in standard logging when the proxy/router rewrites response.model to a model-group alias while custom pricing is on.

get_model_cost_information now passes base_model into cost-name selection when custom pricing is enabled, so lookup does not rely only on the alias in the response object. When building the standard logging payload, if base_model is missing it falls back to metadata["deployment"] (the real deployment id the router records).

Debug printing of standard logging payloads now flushes stdout so piped logging shows output promptly. New tests cover custom-pricing base-model selection and the deployment fallback.

^{Reviewed by Cursor Bugbot for commit 8fabf6f. Bugbot is set up for automated code reviews on this repo. Configure here.}

Use deployment model for standard logging cost-map lookup when the router overrides response.model to a group alias, and flush stdout when printing the payload. Co-authored-by: Cursor <cursoragent@cursor.com>

greptile-apps · 2026-07-02T05:42:44Z

Greptile Summary

This PR fixes a bug where model_map_value was null in StandardLoggingPayload when a proxy model-group alias overrode response.model and custom pricing was enabled. It also adds a metadata["deployment"] fallback for base_model resolution and flushes stdout in the debug print helper.

get_model_cost_information fix (line 4725): Changes model=None to model=base_model if custom_pricing else None when calling _select_model_name_for_cost_calc, so the deployment model name — not the aliased response model — drives the cost-map lookup under custom pricing.
Deployment fallback (lines 5274–5275): When _get_base_model_from_metadata returns None, unconditionally falls back to metadata.get("deployment") as base_model; this applies to all calls, not only custom-pricing ones.
Tests: Adds two new tests covering both changed code paths, addressing the coverage gap noted in prior review.

Confidence Score: 5/5

The changes are narrowly scoped to the logging payload builder and do not touch request handling, authentication, or cost calculation itself — safe to merge.

The core fix in get_model_cost_information is logically correct: passing base_model as model when custom_pricing=True ensures the deployment model name is used for cost-map lookup rather than the aliased response model. The deployment fallback and the flush=True change are both low-risk. Both new code paths now have test coverage. No new regressions are introduced on the standard (non-custom-pricing) path.

No files require special attention beyond what was already raised in prior review threads.

Important Files Changed

Filename	Overview
litellm/litellm_core_utils/litellm_logging.py	Two targeted changes: (1) passes `base_model` as the `model` arg to `_select_model_name_for_cost_calc` when `custom_pricing=True`, fixing a null `model_map_value`; (2) unconditionally falls back `base_model` to `metadata["deployment"]` when the metadata helper returns None; plus a minor `flush=True` on the debug print.
tests/logging_callback_tests/test_standard_logging_payload.py	Adds two new tests: one unit test for the `get_model_cost_information` custom-pricing fix, and one integration-style test exercising the `metadata["deployment"]` fallback path end-to-end in `get_standard_logging_object_payload`.

_{Reviews (6): Last reviewed commit: "Merge branch 'litellm_internal_staging' ..." | Re-trigger Greptile}

codecov · 2026-07-02T05:44:42Z

Codecov Report

❌ Patch coverage is 66.66667% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/litellm_core_utils/litellm_logging.py	66.66%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

…d logging payload Address review: explain why the metadata["deployment"] fallback is unconditional, and add a test covering the get_standard_logging_object_payload code path. Co-authored-by: Cursor <cursoragent@cursor.com>

Sameerlite · 2026-07-02T05:50:16Z

@greptileai

Co-authored-by: Cursor <cursoragent@cursor.com>

Sameerlite · 2026-07-02T06:27:49Z

@greptileai

Passing model=base_model unconditionally caused _get_provider_for_cost_calc to infer and prepend a provider prefix on all non-custom-pricing calls, changing model_map_key for existing deployments. Scope it to custom_pricing=True where the fix is actually needed. Co-authored-by: Cursor <cursoragent@cursor.com>

mateo-berri · 2026-07-02T06:28:45Z

@greptileai

Generated by Claude Code

…ging_model_map

mateo-berri · 2026-07-02T06:33:25Z

@greptileai

Generated by Claude Code

mateo-berri · 2026-07-02T06:34:36Z

bugbot run

Generated by Claude Code

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 8fabf6f. Configure here.}

mateo-berri

LGTM; thanks!

…AI#31940) * fix(logging): resolve model_map_value for proxy custom pricing Use deployment model for standard logging cost-map lookup when the router overrides response.model to a group alias, and flush stdout when printing the payload. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(logging): add comment and test for deployment fallback in standard logging payload Address review: explain why the metadata["deployment"] fallback is unconditional, and add a test covering the get_standard_logging_object_payload code path. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(test): update model_map_key assertion for provider-prefixed keys Co-authored-by: Cursor <cursoragent@cursor.com> * fix(logging): scope base_model to model param only under custom_pricing Passing model=base_model unconditionally caused _get_provider_for_cost_calc to infer and prepend a provider prefix on all non-custom-pricing calls, changing model_map_key for existing deployments. Scope it to custom_pricing=True where the fix is actually needed. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com>

fix(logging): resolve model_map_value for proxy custom pricing

862f4d5

Use deployment model for standard logging cost-map lookup when the router overrides response.model to a group alias, and flush stdout when printing the payload. Co-authored-by: Cursor <cursoragent@cursor.com>

greptile-apps Bot reviewed Jul 2, 2026

View reviewed changes

Comment thread litellm/litellm_core_utils/litellm_logging.py

Comment thread tests/logging_callback_tests/test_standard_logging_payload.py

Sameerlite marked this pull request as draft July 2, 2026 06:23

fix(test): update model_map_key assertion for provider-prefixed keys

d9b4eaf

Co-authored-by: Cursor <cursoragent@cursor.com>

mateo-berri marked this pull request as ready for review July 2, 2026 06:26

Merge branch 'litellm_internal_staging' into litellm_fix_standard_log…

8fabf6f

…ging_model_map

cursor Bot reviewed Jul 2, 2026

View reviewed changes

mateo-berri approved these changes Jul 2, 2026

View reviewed changes

mateo-berri merged commit 8d0dc92 into litellm_internal_staging Jul 2, 2026
123 of 124 checks passed

mateo-berri deleted the litellm_fix_standard_logging_model_map branch July 2, 2026 15:07

claude Bot mentioned this pull request Jul 3, 2026

fix(container): update image ghcr.io/berriai/litellm-non_root (v1.90.2 ➔ v1.90.3) drag0n141/home-ops#6663

Merged

1 task

Navigation Menu

Uh oh!

Uh oh!

fix(logging): resolve model_map_value for proxy custom pricing - #31940

fix(logging): resolve model_map_value for proxy custom pricing#31940
mateo-berri merged 5 commits into
litellm_internal_stagingfrom
litellm_fix_standard_logging_model_map

Sameerlite commented Jul 2, 2026 •

edited by cursor Bot

Loading

Uh oh!

greptile-apps Bot commented Jul 2, 2026 •

edited

Loading

Important Files Changed

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Jul 2, 2026

Uh oh!

Sameerlite commented Jul 2, 2026

Uh oh!

Sameerlite commented Jul 2, 2026

Uh oh!

mateo-berri commented Jul 2, 2026

Uh oh!

mateo-berri commented Jul 2, 2026

Uh oh!

mateo-berri commented Jul 2, 2026

Uh oh!

cursor Bot left a comment

Uh oh!

mateo-berri left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

Sameerlite commented Jul 2, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

greptile-apps Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Jul 2, 2026

Codecov Report

Uh oh!

Sameerlite commented Jul 2, 2026

Uh oh!

Sameerlite commented Jul 2, 2026

Uh oh!

mateo-berri commented Jul 2, 2026

Uh oh!

mateo-berri commented Jul 2, 2026

Uh oh!

mateo-berri commented Jul 2, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

mateo-berri left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Sameerlite commented Jul 2, 2026 •

edited by cursor Bot

Loading

greptile-apps Bot commented Jul 2, 2026 •

edited

Loading