docs(headroom): document Headroom prompt compression integration - #420
Conversation
Adds a new page covering the Headroom integration that ships as a pre-call LiteLLM guardrail plugin. Covers config, opt-in via key or request body, Claude Code rollout flow, validation in the Logs UI, deploy via Dockerfile, and the headroom service contract. Sidebars surface it under Cost Optimization in both the proxy and integrations sidebars; the SDK prompt_compression page links across to the proxy variant.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5e7f89487f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| In the Admin UI, open any request in **Logs**, scroll to the **Guardrails & Policy Compliance** panel, and you will see `headroom-compression` listed under **Request Lifecycle** as a `pre-call` step with its latency, plus an entry under **Evaluation Details**. | ||
|
|
||
|  |
There was a problem hiding this comment.
When users follow this validation section, the referenced image shows the request status as Failure, 0 Passed, and Pre-call guardrail: headroom-compression FAILED, so it demonstrates a failed compression attempt rather than a successful Headroom run. This can lead operators to treat a failed guardrail as the expected validation state; please use a screenshot from a successful run or update the text to explain failure diagnostics instead.
Useful? React with 👍 / 👎.
2 participants
Summary
Adds a new doc for the Headroom integration introduced in BerriAI/litellm#31407. Headroom ships as a pre-call LiteLLM guardrail plugin but is prompt compression, not safety filtering, so the page is positioned under Cost Optimization rather than Guardrail Providers.
The doc covers:
config.yamland the underlyingPOST /v1/compresscontract Headroom exposes.default_on: truefor always-on, virtual key withguardrails: ["headroom-compression"]for per-user rollout, and request-body opt-in (guardrailsarray on/v1/chat/completions,litellm_metadata.guardrailson/v1/messages).ANTHROPIC_BASE_URLandANTHROPIC_AUTH_TOKENsetup and thex-headroom-bypass: trueper-request escape hatch.Sidebar changes add the page under Cost Optimization in both the proxy and integrations sidebars, and the existing SDK
prompt_compressionpage links across to the proxy variant.Test plan
npm run startrenders/docs/proxy/headroomwithout MDX errors/img/headroom_logs.pngrenders inline