fix(arize): _set_usage_outputs handles raw OpenAI Pydantic CompletionUsage

[alvinttang]

Summary

Fixes #13672 (252 days old, 18 comments). _set_usage_outputs in litellm/integrations/arize/_utils.py calls usage.get("total_tokens") and usage.get("output_tokens_details", {}).get("reasoning_tokens"). When the logger is invoked via langfuse_otel (or arize on a Chat Completions response wrapped as raw OpenAI Pydantic), usage is openai.types.completion_usage.CompletionUsage — a Pydantic v2 model with no .get method — so the call raises AttributeError. Same for nested CompletionTokensDetails / OutputTokensDetails.

Latent secondary bug: reasoning_tokens for Chat Completions live in completion_tokens_details, not output_tokens_details, so they were silently dropped on that path.

Fix

_safe_get(obj, key, default) helper that prefers dict-style .get when callable and falls back to getattr otherwise — uniform for dicts, litellm Usage, and raw OpenAI Pydantic models. Used for total / completion / prompt / output tokens. For reasoning tokens, tries completion_tokens_details (Chat Completions) first, then output_tokens_details (Responses API). +42 LOC.

Test

tests/test_litellm/integrations/arize/test_arize_utils.py:

• test_set_usage_outputs_pydantic_completion_usage — Chat Completions API path with raw CompletionUsage + CompletionTokensDetails.
• test_set_usage_outputs_pydantic_response_api_usage — Responses API path with usage object lacking .get.

Both fail on main with AttributeError, pass after the fix. Wider pytest of arize + langfuse integrations: 52/52 (2 pre-existing proxy-import failures unrelated). ruff clean.

Risk notes

• Backward-compatible: _safe_get returns identical values for dicts and litellm Usage (which has .get); only differs on Pydantic-without-.get, where it now succeeds instead of crashing.
• The completion_tokens_details lookup change is additive — adds reasoning_tokens for Chat Completions that were previously silently dropped; does not regress Responses API.
• Different subsystem from existing user PRs.

Refs #13672

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 4 committers have signed the CLA.

✅ yuneng-berri
✅ mateo-berri
❌ alvinttang
❌ krrish-berri-2

---

alvinttang seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[veria-ai]

Low: No security issues

This PR adds a _safe_get helper to handle Pydantic models without .get() in the Arize/Langfuse OTel usage logger, plus new Azure model pricing entries. All changes are internal telemetry and static config — no security-relevant code paths are affected.

---

Status: 0 open
Risk: 1/10

Posted by Veria AI · 2026-04-25T21:08:53.488Z

[greptile-apps]

Greptile Summary

This PR fixes a long-standing AttributeError in _set_usage_outputs when the usage object is a raw OpenAI Pydantic model (e.g. CompletionUsage) that lacks a .get() method. It also corrects the reasoning_tokens lookup path for Chat Completions (completion_tokens_details vs output_tokens_details), and adds azure/gpt-5.5 model family entries plus a new supports_low_reasoning_effort schema field. The fix is narrow, backward-compatible, and well-tested.

Confidence Score: 5/5

Safe to merge — the fix is backward-compatible, narrowly scoped, and fully covered by new regression tests.

No P0 or P1 findings. The _safe_get helper is a clean, defensively-written shim; both dict and pydantic paths return identical values to the previous code for objects that already had .get. The or-based truthiness concern on the token_details lookup and the fragile precondition assertion were already flagged in prior review threads. JSON additions follow established schema conventions.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/integrations/arize/_utils.py	Adds _safe_get helper to unify dict/pydantic attribute access and updates _set_usage_outputs to use it; also fixes reasoning_tokens lookup order for Chat vs Responses API.
tests/test_litellm/integrations/arize/test_arize_utils.py	Adds two regression tests covering the Pydantic CompletionUsage and Responses API paths; no real network calls made.
tests/test_litellm/test_utils.py	Extends the JSON schema validator to allow supports_low_reasoning_effort as a valid boolean field.
litellm/model_prices_and_context_window_backup.json	Adds azure/gpt-5.5, azure/gpt-5.5-2026-04-23, and azure/gpt-5.5-pro model entries with pricing, context window, and capability flags.
model_prices_and_context_window.json	Mirror of backup JSON: adds same azure/gpt-5.5 model family entries.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["_set_usage_outputs(span, response_obj, span_attrs)"] --> B["usage = response_obj.get('usage')"]
    B --> C{usage is falsy?}
    C -- yes --> Z[return]
    C -- no --> D["_safe_get(usage, 'total_tokens')"]
    D --> E["set LLM_TOKEN_COUNT_TOTAL"]
    E --> F["_safe_get(usage, 'completion_tokens')\n OR _safe_get(usage, 'output_tokens')"]
    F --> G["set LLM_TOKEN_COUNT_COMPLETION"]
    G --> H["_safe_get(usage, 'prompt_tokens')\n OR _safe_get(usage, 'input_tokens')"]
    H --> I["set LLM_TOKEN_COUNT_PROMPT"]
    I --> J["token_details = _safe_get(usage, 'completion_tokens_details')\n OR _safe_get(usage, 'output_tokens_details')"]
    J --> K["_safe_get(token_details, 'reasoning_tokens')"]
    K --> L["set LLM_TOKEN_COUNT_COMPLETION_DETAILS_REASONING"]

    subgraph _safe_get["_safe_get(obj, key, default)"]
        S1{obj is None?} -- yes --> S2[return default]
        S1 -- no --> S3{obj has callable .get?}
        S3 -- yes --> S4["obj.get(key, default)"]
        S3 -- no --> S5["getattr(obj, key, default)"]
    end

Loading

_{Reviews (2): Last reviewed commit: "Merge branch 'litellm-oss-staging-04-25-..." | Re-trigger Greptile}

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing alvinttang:fix/arize-langfuse-otel-pydantic-usage (a74772c) with main (0beec45)}

[greptile-apps]

or truthiness check may bypass completion_tokens_details unintentionally

If a CompletionTokensDetails Pydantic model instance defines a custom __bool__ or __len__ that returns falsy (e.g. all fields are zero/None), _safe_get(usage, "completion_tokens_details") or _safe_get(usage, "output_tokens_details") would fall through to output_tokens_details even though completion_tokens_details was explicitly present. Using an explicit None check would be safer:

token_details = _safe_get(usage, "completion_tokens_details")
if token_details is None:
    token_details = _safe_get(usage, "output_tokens_details")

Suggested change

	token_details = _safe_get(usage, "completion_tokens_details") or _safe_get(
	usage, "output_tokens_details"
	)
	token_details = _safe_get(usage, "completion_tokens_details")
	if token_details is None:
	token_details = _safe_get(usage, "output_tokens_details")

[greptile-apps]

Precondition assertion is fragile against library updates

assert not hasattr(usage, "get") hard-codes an assumption about the openai SDK's internal Pydantic model structure. If a future version of openai or Pydantic v2 adds a .get() method (e.g. for dict-compatibility), this assertion will fail and block CI even though the fix itself still works correctly via the callable(getter) branch of _safe_get. Consider removing this precondition or rewriting it as a pytest.skip with a note, so the test degrades gracefully.

[codecov]

Codecov Report

❌ Patch coverage is 85.71429% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/integrations/arize/_utils.py	85.71%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!