Skip to content

fix(headroom guardrail): log real token/compression stats instead of "allow" - #32153

Closed
krrish-berri-2 wants to merge 3 commits into
BerriAI:litellm_internal_stagingfrom
krrish-berri-2:litellm_fix_headroom_guardrail_token_logging
Closed

fix(headroom guardrail): log real token/compression stats instead of "allow"#32153
krrish-berri-2 wants to merge 3 commits into
BerriAI:litellm_internal_stagingfrom
krrish-berri-2:litellm_fix_headroom_guardrail_token_logging

Conversation

@krrish-berri-2

Copy link
Copy Markdown
Contributor

(Replaces #32102, closed by GitHub when the head branch was renamed to match repo convention.)

Summary

  • The headroom guardrail already fetches tokens_before/tokens_after/compression_ratio/transforms_applied from Headroom's /v1/compress response, but only logged them via a debug-level verbose_proxy_logger.debug(...) call - never persisted anywhere.
  • As a result, spend_logs.guardrail_information.guardrail_response always showed the generic "allow", with no way to tell whether compression actually ran or by how much.
  • _call_compress now returns the token/compression stats alongside the compressed messages and success flag, and apply_guardrail logs them via add_standard_logging_guardrail_information_to_request_data when compression succeeds (the same mechanism other guardrails like block_code_execution and cisco_ai_defense use for rich logging).
  • Raw message content is intentionally excluded from what's logged - only token counts, compression ratio, and applied transform names, so no request/response text lands in spend logs.
  • Rebased on top of litellm_internal_staging's existing fail_open/fail_closed unreachable-fallback change to the same function - stats are only logged on the success path.

Before:

"guardrail_response": "allow"

After:

"guardrail_response": {
  "tokens_before": 1833,
  "tokens_after": 1429,
  "tokens_saved": 404,
  "compression_ratio": 0.78,
  "transforms_applied": ["router:mixed:0.44"]
}

Test plan

  • Verified locally against a running headroom-ai 0.28.0 proxy + litellm proxy with the headroom guardrail configured
  • Ran real /v1/chat/completions requests through litellm with small/medium/large conversation payloads (plain, JSON tool output, log dumps, diffs, code) and confirmed guardrail_information in LiteLLM_SpendLogs now shows real token stats instead of "allow" for compressed requests
  • Confirmed small/uncompressed requests still log correctly (tokens_saved: 0, compression_ratio: 1.0)

…"allow"

The headroom guardrail fetched tokens_before/tokens_after/compression_ratio
from Headroom's /v1/compress response but only surfaced them via a debug-level
log line, so spend_logs.guardrail_information showed guardrail_response:
"allow" with no way to tell whether compression actually ran or by how much.

_call_compress now returns the token/compression stats alongside the
compressed messages and success flag, and apply_guardrail logs them via
add_standard_logging_guardrail_information_to_request_data when compression
succeeds. Raw message content is intentionally excluded from what's logged -
only token counts, compression ratio, and applied transform names.
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps

greptile-apps Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR surfaces real token and compression statistics from the Headroom /v1/compress response into spend_logs.guardrail_information, replacing the generic "allow" placeholder that was previously always written.

  • _call_compress now returns a third element — a dict of tokens_before, tokens_after, tokens_saved, compression_ratio, and transforms_applied — built by filtering the API body to just those keys so no message content is included.
  • apply_guardrail calls add_standard_logging_guardrail_information_to_request_data with that dict on the success path, using the same mechanism already used by block_code_execution and cisco_ai_defense; the @log_guardrail_information decorator's entries_before snapshot correctly suppresses double-recording.

Confidence Score: 5/5

Safe to merge. The change is narrow, touches only the headroom guardrail, and follows an established pattern already used by other guardrails in the repo.

The only file touched is the headroom guardrail hook. The decorator's double-recording guard (entries_before snapshot) was explicitly designed for this pattern, and the failure paths are unchanged — they still return (messages, False, {}) so no stats are logged when compression doesn't succeed. No message content reaches the spend log.

No files require special attention.

Important Files Changed

Filename Overview
litellm/proxy/guardrails/guardrail_hooks/headroom/headroom.py Changes _call_compress return type to include a stats dict and wires it into add_standard_logging_guardrail_information_to_request_data on the success path; decorator's double-recording guard handles the integration correctly.

Reviews (3): Last reviewed commit: "fix(review): remove comment per repo's n..." | Re-trigger Greptile

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d0ec5b4169

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread litellm/proxy/guardrails/guardrail_hooks/headroom/headroom.py Outdated
@codecov

codecov Bot commented Jul 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.71429% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...xy/guardrails/guardrail_hooks/headroom/headroom.py 85.71% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

@krrish-berri-2

Copy link
Copy Markdown
Contributor Author

@greptile review

…ntion

Addresses codex review feedback - CLAUDE.md says not to add comments
unless explicitly asked; the sensitive-logging guarantee is already
expressed by the stats dict only pulling specific keys, not messages.
@krrish-berri-2

Copy link
Copy Markdown
Contributor Author

@greptile review

@krrish-berri-2

Copy link
Copy Markdown
Contributor Author

Replaced by #32158 - this PR's head branch lived in a fork, which prevented CircleCI from running. Same branch, pushed directly into this repo instead.

auto-merge was automatically disabled July 4, 2026 23:31

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants