Skip to content

fix(a2a): populate response usage in a2a chat transformation - #31980

Merged
Sameerlite merged 2 commits into
litellm_internal_stagingfrom
litellm_fix_a2a_chat_usage
Jul 3, 2026
Merged

fix(a2a): populate response usage in a2a chat transformation#31980
Sameerlite merged 2 commits into
litellm_internal_stagingfrom
litellm_fix_a2a_chat_usage

Conversation

@michelligabriele

Copy link
Copy Markdown
Collaborator

Relevant issues

Linear ticket

Resolves LIT-3291

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Called the a2a/ provider through /chat/completions on a local proxy configured with per-token pricing for the model:

curl -s -X POST http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{"model":"echo-a2a-chat","messages":[{"role":"user","content":"Say hello in one word"}]}'

Before fix:

{"choices": [...], "usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0}}

After fix:

{"choices": [...], "usage": {"prompt_tokens": 12, "completion_tokens": 9, "total_tokens": 21}}

usage is no longer 0/0/0, and spend logs now show real cost computed from the configured input_cost_per_token / output_cost_per_token against this usage (previously spend was $0.00 because cost is priced off a zero token count).

Type

🐛 Bug Fix

Changes

The a2a/ chat-completions provider (litellm/llms/a2a/chat/transformation.py) never set usage on the transformed response, so callers of model=a2a/<agent> always got usage: {prompt_tokens: 0, completion_tokens: 0, total_tokens: 0} directly in the API response, and per-token cost always computed to $0.00 since it prices off that zero usage.

A2A agents don't return token usage in their protocol responses, so this estimates it with litellm.utils.token_counter off the request messages and response text, following the same pattern already used by the langgraph and langflow agent-bridge providers for the same "agent doesn't report usage" situation.

@greptile-apps

greptile-apps Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes the A2A chat provider always returning usage: {prompt_tokens: 0, completion_tokens: 0, total_tokens: 0}, which caused per-token cost calculations to always produce $0.00. Since A2A agents don't include token counts in their protocol responses, the fix estimates usage via litellm.utils.token_counter (using the gpt-3.5-turbo tokenizer as a proxy), the same approach already used by the langgraph, vertex_ai/agent_engine, and azure_ai/agents providers.

  • litellm/llms/a2a/chat/transformation.py: Adds a best-effort try/except block after the response is built that estimates prompt_tokens from the input messages and completion_tokens from the response text, then sets a Usage object on model_response.
  • tests/test_litellm/llms/a2a/chat/test_a2a_chat_transformation.py: New mock-only regression test that asserts all three token counts are non-zero and that total_tokens equals the sum of the other two.

Confidence Score: 5/5

Safe to merge — the change is additive, wrapped in a best-effort try/except so a tokenizer failure cannot break the response path, and it exactly mirrors the pattern already in production for three other agent bridge providers.

The fix is a small, well-scoped addition that follows a pattern proven across three other providers in the same codebase. The estimation runs inside a try/except so any edge-case failure degrades gracefully to the existing 0/0/0 behaviour rather than raising. The new test is mock-only and validates the core regression directly.

No files require special attention.

Important Files Changed

Filename Overview
litellm/llms/a2a/chat/transformation.py Adds best-effort token usage estimation via token_counter, following the identical pattern used by langgraph, vertex_ai/agent_engine, and azure_ai/agents providers. Usage and import are correct.
tests/test_litellm/llms/a2a/chat/test_a2a_chat_transformation.py New regression test using mocks only; correctly verifies that prompt_tokens, completion_tokens, and total_tokens are all non-zero and self-consistent after the fix.

Reviews (2): Last reviewed commit: "test(a2a): rename to avoid module basena..." | Re-trigger Greptile

Comment on lines +331 to +332
except Exception: # noqa: BLE001 - best-effort estimate; a tokenizer hiccup must not break the response
pass

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The bare except Exception: pass silently swallows any token-counting failure with no log output, making it impossible to diagnose why usage stays at zero for a user. Both langgraph/chat/transformation.py and vertex_ai/agent_engine/transformation.py use the same pattern but emit a verbose_logger.warning so the failure is at least visible in debug logs.

Suggested change
except Exception: # noqa: BLE001 - best-effort estimate; a tokenizer hiccup must not break the response
pass
except Exception as e: # noqa: BLE001 - best-effort estimate; a tokenizer hiccup must not break the response
verbose_logger.warning(f"A2A: failed to estimate token usage: {e}")

@codecov

codecov Bot commented Jul 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.00000% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
litellm/llms/a2a/chat/transformation.py 75.00% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

@Sameerlite

Copy link
Copy Markdown
Contributor

@greptileai

@Sameerlite Sameerlite left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Sameerlite
Sameerlite merged commit cbe23d6 into litellm_internal_staging Jul 3, 2026
122 checks passed
@Sameerlite
Sameerlite deleted the litellm_fix_a2a_chat_usage branch July 3, 2026 03:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants