fix(headroom guardrail): log real token/compression stats instead of "allow" - #32153
Conversation
…"allow" The headroom guardrail fetched tokens_before/tokens_after/compression_ratio from Headroom's /v1/compress response but only surfaced them via a debug-level log line, so spend_logs.guardrail_information showed guardrail_response: "allow" with no way to tell whether compression actually ran or by how much. _call_compress now returns the token/compression stats alongside the compressed messages and success flag, and apply_guardrail logs them via add_standard_logging_guardrail_information_to_request_data when compression succeeds. Raw message content is intentionally excluded from what's logged - only token counts, compression ratio, and applied transform names.
|
|
Greptile SummaryThis PR surfaces real token and compression statistics from the Headroom
Confidence Score: 5/5Safe to merge. The change is narrow, touches only the headroom guardrail, and follows an established pattern already used by other guardrails in the repo. The only file touched is the headroom guardrail hook. The decorator's double-recording guard ( No files require special attention.
|
| Filename | Overview |
|---|---|
| litellm/proxy/guardrails/guardrail_hooks/headroom/headroom.py | Changes _call_compress return type to include a stats dict and wires it into add_standard_logging_guardrail_information_to_request_data on the success path; decorator's double-recording guard handles the integration correctly. |
Reviews (3): Last reviewed commit: "fix(review): remove comment per repo's n..." | Re-trigger Greptile
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d0ec5b4169
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
@greptile review |
…ntion Addresses codex review feedback - CLAUDE.md says not to add comments unless explicitly asked; the sensitive-logging guarantee is already expressed by the stats dict only pulling specific keys, not messages.
|
@greptile review |
|
Replaced by #32158 - this PR's head branch lived in a fork, which prevented CircleCI from running. Same branch, pushed directly into this repo instead. |
Pull request was closed
(Replaces #32102, closed by GitHub when the head branch was renamed to match repo convention.)
Summary
tokens_before/tokens_after/compression_ratio/transforms_appliedfrom Headroom's/v1/compressresponse, but only logged them via a debug-levelverbose_proxy_logger.debug(...)call - never persisted anywhere.spend_logs.guardrail_information.guardrail_responsealways showed the generic"allow", with no way to tell whether compression actually ran or by how much._call_compressnow returns the token/compression stats alongside the compressed messages and success flag, andapply_guardraillogs them viaadd_standard_logging_guardrail_information_to_request_datawhen compression succeeds (the same mechanism other guardrails likeblock_code_executionandcisco_ai_defenseuse for rich logging).litellm_internal_staging's existingfail_open/fail_closedunreachable-fallback change to the same function - stats are only logged on the success path.Before:
After:
Test plan
/v1/chat/completionsrequests through litellm with small/medium/large conversation payloads (plain, JSON tool output, log dumps, diffs, code) and confirmedguardrail_informationinLiteLLM_SpendLogsnow shows real token stats instead of"allow"for compressed requeststokens_saved: 0,compression_ratio: 1.0)