fix(logging): backfill streaming hidden response cost

[milan-berri]

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Screenshots / Proof of Fix

Validated with LiteLLM proxy configured with callbacks: ["otel"] and Jaeger as the OTLP backend.

After the fix:

• Non-streaming request:

  • llm.is_streaming: False
  • gen_ai.cost.total_cost: 4.6499999999999995e-06
  • hidden_params.response_cost: 4.6499999999999995e-06

• Streaming request:

  • llm.is_streaming: True
  • gen_ai.cost.total_cost: 3.9e-06
  • hidden_params.response_cost: 3.9e-06

The streaming value is now present on the main Received Proxy Server Request span, which reads from standard_logging_object.hidden_params.

Test run:

python -m pytest tests/test_litellm/litellm_core_utils/test_litellm_logging.py -v

Result:

73 passed

Type

🐛 Bug Fix
✅ Test

Changes

• Backfill litellm_params.metadata.hidden_params.response_cost for assembled streaming responses using the already-calculated model_call_details["response_cost"].
• Backfill standard_logging_object.hidden_params.response_cost from the same calculated cost so OTEL/Jaeger spans include the streaming cost.
• Preserve existing non-null provider-supplied hidden_params.response_cost.
• Avoid mutating the original response object's _hidden_params while constructing logging payloads.
• Add unit tests covering metadata backfill, standard logging payload backfill, no-overwrite behavior, and non-mutation.

[greptile-apps]

Greptile Summary

This PR fixes a gap where streaming responses did not expose the calculated response_cost in litellm_params.metadata.hidden_params or standard_logging_payload.hidden_params, causing OTEL/Jaeger spans and other callbacks to see a null cost for streamed calls. The fix backfills the already-calculated cost into both locations while preserving any provider-supplied cost and avoiding mutation of the original response object's _hidden_params.

Confidence Score: 5/5

Safe to merge — targeted backfill with no mutation of existing objects and no breaking changes.

No P0 or P1 issues found. Both backfill paths are correctly guarded (only fills when None, preserves provider-supplied cost), call sites confirm response_cost is calculated before the merge function runs, the .copy() correctly prevents mutation of the original _hidden_params, and all new tests are mock-based and meaningful.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/litellm_core_utils/litellm_logging.py	Two targeted changes: _merge_hidden_params_from_response_into_metadata now uses .copy() and backfills response_cost; get_standard_logging_object_payload moves clean_hidden_params setup after saved_cache_cost and conditionally backfills response_cost. Both call sites confirm cost is set before the merge runs.
litellm/types/utils.py	Corrects StandardLoggingHiddenParams.response_cost type from Optional[str] to Optional[Union[str, float]] to accommodate the float cost values now being stored there.
tests/test_litellm/litellm_core_utils/test_litellm_logging.py	Adds four new unit tests covering: backfill when cost is null, no-overwrite when cost is already set, non-mutation of the original response object, and end-to-end standard logging payload backfill. No network calls; all tests are mock-based.

Sequence Diagram

sequenceDiagram
    participant SC as Streaming Chunk Handler
    participant Log as Logging.success_handler
    participant Merge as _merge_hidden_params_from_response_into_metadata
    participant Build as _build_standard_logging_payload
    participant SLP as get_standard_logging_object_payload

    SC->>Log: Stream complete
    Log->>Log: _response_cost_calculator(complete_streaming_response)
    Log->>Merge: _merge_hidden_params_from_response_into_metadata(response)
    Merge->>Merge: hidden_params = response._hidden_params.copy()
    Merge->>Merge: if hidden_params[response_cost] is None and model_call_details[response_cost] is not None backfill cost
    Merge->>Log: metadata[hidden_params][response_cost] = cost
    Log->>Build: _build_standard_logging_payload(response, ...)
    Build->>SLP: get_standard_logging_object_payload(kwargs, response, ...)
    SLP->>SLP: hidden_params = response._hidden_params
    SLP->>SLP: clean_hidden_params = get_hidden_params(hidden_params)
    SLP->>SLP: if clean_hidden_params[response_cost] is None and raw_response_cost is not None backfill cost
    SLP->>Log: StandardLoggingPayload with hidden_params.response_cost = cost

Loading

_{Reviews (2): Last reviewed commit: "refactor(logging): simplify hidden respo..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!