feat(guardrails): add send_images and exclude_payload_fields to generic_guardrail_api by caduri · Pull Request #43755 · BerriAI/litellm

caduri · 2026-09-29T19:30:32Z

TLDR

Problem this solves:

Every generic guardrail call carries full images, even for text-only guardrails
There is no way to drop request fields a guardrail never reads

How it solves it:

send_images: false withholds image data from the guardrail call
Image parts stay in place with the URL replaced by [omitted]
exclude_payload_fields drops named top-level fields from the guardrail call
A rewrite of content the guardrail was not sent is never applied silently
An invalid value is ignored with a warning and the default is kept

Relevant issues

Split out of #37055. Depends on #43679, so review only the top commit until it merges. Docs: BerriAI/litellm-docs#2075

User Flow

Before: an admin whose guardrail only reads text has every image and header sent to it on every request

The admin opens https://litellm-domain/ui/?page=guardrails, clicks Add New Guardrail, picks Generic Guardrail API and clicks Next
The form ends at streaming_transform_mode, with no option to stop sending images
A developer sends POST https://litellm-domain/v1/chat/completions with text plus a base64 image_url part, or the same content to /v1/responses or /v1/messages
The guardrail endpoint receives the full image inside structured_messages, plus request_headers, tools and every other field

After: the same guardrail gets only what it needs, and the model still sees the image

The admin opens the same form and now sees send_images as its last field, or sets send_images: false and exclude_payload_fields: ["request_headers", "tools"] in config.yaml
On restart the proxy log warns which fields are not sent to the guardrail
The developer sends the same POST, and the guardrail receives no images, request_headers or tools, with each image URL reading [omitted]
The model still gets the original image and answers about it
If the guardrail answers with a rewrite of a field it was not sent, the request or response is rejected instead of passing through unrewritten

The screenshots were taken at 3465bfb, and no commit since then touches the Admin UI form. To reproduce them, open http://localhost:4000/ui/?page=guardrails, click Add New Guardrail, then Add Provider Guardrail, pick Generic Guardrail API, type any name, click Next and scroll the form to the bottom. exclude_payload_fields is config-only, so the form does not show it

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have added meaningful tests
- A regression test pins the bypass Greptile found: with texts excluded, a guardrail that masks them and echoes Anthropic rows without their null fields is now rejected, and the test fails if fix(guardrails): keep masked text when a guardrail echoes multipart rows #43679's echo fix is reverted
- Each of these 26 mutations fails a test: withholding images when send_images is unset or invalid, parsing send_images strictly, keeping unknown or protected field names in the exclude list, still posting excluded fields, leaving image URLs inside messages or the top-level images field when images are withheld, skipping the refusal check or running it for every action, counting returned images as an applied change, not counting returned texts, tools or written-back rows, counting tools or rows the endpoint never posted, treating an unchanged echo as a change, accepting rewrites to fields that were not sent, not restoring the caller's image into an echoed placeholder part, accepting a newly added placeholder, raising on an invalid exclude list, showing config-only fields in the Admin UI form, not forwarding either option from the config, and naming the wrong side in the refusal message
The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
- Every check passes on d761026, including code-quality and codecov/patch
- make check passes locally with BASE_REF=upstream/main. A fork whose main lags upstream fails the budget gates against that stale base, so set BASE_REF when checking a fork
- CodeQL flags _send_images and image_part_url for mixing implicit and explicit returns. Both are false positives: every path through the try/except returns, and the match ends in case _: return None
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)
- Greptile scored 5/5 on 3465bfb. Its later review of feat(guardrails): add max_messages, max_text_chars and strip_patterns to generic_guardrail_api #43761 flagged a masking bypass in the row echo check, fixed in fix(guardrails): keep masked text when a guardrail echoes multipart rows #43679, and asked for the sentinels to live in constants.py, done in d761026. Greptile scored 5/5 on d761026, and its one note, repeated row serialization on large image rows, is answered in the thread with a measurement (about 0.06s for a 20 MB row)
- Veria and Bugbot have not run on this PR

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

All runs are against a live proxy on localhost:4000 started with python litellm/proxy/proxy_cli.py --config proof_config.yaml --detailed_debug --use_v2_migration_resolver, backed by a real Postgres, and every completion is a real, billed call to Bedrock us.anthropic.claude-haiku-4-5-20251001-v1:0. $PROOF_KEY is a virtual key from POST /key/generate with access to bedrock-haiku-4-5, and $RED is a base64 PNG of a solid red square. Each request picks its guardrail with "guardrails": [...]

model_list:
  - model_name: bedrock-haiku-4-5
    litellm_params:
      model: bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
      aws_region_name: us-east-1

guardrails:
  - guardrail_name: no-images
    litellm_params:
      guardrail: generic_guardrail_api
      mode: pre_call
      api_base: http://127.0.0.1:8787
      default_on: false
      send_images: false
  - guardrail_name: no-headers
    litellm_params:
      guardrail: generic_guardrail_api
      mode: pre_call
      api_base: http://127.0.0.1:8787
      default_on: false
      exclude_payload_fields: ["request_headers", "tools"]
  - guardrail_name: no-texts
    litellm_params:
      guardrail: generic_guardrail_api
      mode: pre_call
      api_base: http://127.0.0.1:8787
      default_on: false
      exclude_payload_fields: ["texts"]
  - guardrail_name: no-texts-response
    litellm_params:
      guardrail: generic_guardrail_api
      mode: post_call
      api_base: http://127.0.0.1:8787
      default_on: false
      exclude_payload_fields: ["texts"]
  - guardrail_name: typo
    litellm_params:
      guardrail: generic_guardrail_api
      mode: pre_call
      api_base: http://127.0.0.1:8787
      default_on: false
      send_images: "maybe"

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL
  proxy_batch_write_at: 1

At the base commit neither option exists, so the proxy ignores both and sends the guardrail everything. The guardrail endpoint is a small FastAPI app on 127.0.0.1:8787 that logs which fields it received, how many top-level images it got, and the first 40 characters of every image URL inside structured_messages. It allows everything, except that when texts is missing it answers GUARDRAIL_INTERVENED with masked texts, which is the rewrite a PII guardrail would send. On the response side it also returns a tool list, which no response handler applies. Nothing else is stubbed

import json
from pathlib import Path

from fastapi import FastAPI, Request

LOG = Path(__file__).with_name("endpoint.jsonl")
app = FastAPI()


def _image_urls(messages: object) -> list[str]:
    urls = []
    for message in messages if isinstance(messages, list) else []:
        content = message.get("content") if isinstance(message, dict) else None
        for part in content if isinstance(content, list) else []:
            image_url = part.get("image_url") if isinstance(part, dict) else None
            url = image_url.get("url") if isinstance(image_url, dict) else image_url
            if isinstance(url, str):
                urls.append(url[:40])
    return urls


@app.post("/beta/litellm_basic_guardrail_api")
async def guardrail(request: Request) -> dict[str, object]:
    payload = await request.json()
    with LOG.open("a") as log:
        log.write(
            json.dumps(
                {
                    "input_type": payload.get("input_type"),
                    "fields_received": sorted(key for key, value in payload.items() if value is not None),
                    "images": len(payload.get("images") or []),
                    "image_urls_in_messages": _image_urls(payload.get("structured_messages")),
                }
            )
            + "\n"
        )
    if "texts" not in payload and payload.get("input_type") == "response":
        return {"action": "GUARDRAIL_INTERVENED", "texts": ["[MASKED]"], "tools": [{"type": "function", "function": {"name": "lookup"}}]}
    if "texts" not in payload:
        return {"action": "GUARDRAIL_INTERVENED", "texts": ["[MASKED]"]}
    return {"action": "NONE"}


@app.post("/_reset")
async def reset() -> dict[str, int]:
    LOG.write_text("")
    return {"records": 0}

The model still answers "Red" with send_images: false, because the setting only changes what the guardrail is sent, never what the model receives

Before (`5b61f8b`)

Case 1: /v1/chat/completions with an image and send_images: false

Run

$ curl -s http://localhost:4000/v1/chat/completions -H "Authorization: Bearer $PROOF_KEY" -H 'Content-Type: application/json' -d '{"model": "bedrock-haiku-4-5", "max_tokens": 10, "guardrails": ["no-images"], "messages": [{"role": "user", "content": [{"type": "text", "text": "What color is this image? Answer with one word"}, {"type": "image_url", "image_url": {"url": "data:image/png;base64,$RED"}}]}]}' | jq -c 'if .error then {error: .error.message} else {answer: .choices[0].message.content} end'

Observe
```
{"answer":"Red"}
```
Run
```
$ jq -c . endpoint.jsonl
```

Observe

{"input_type":"request","fields_received":["additional_provider_specific_params","images","input_type","litellm_call_id","litellm_trace_id","litellm_version","model","request_data","request_headers","structured_messages","texts"],"images":1,"image_urls_in_messages":["data:image/png;base64,iVBORw0KGgoAAAANSU"]}

Case 2: /v1/responses with an input_image and send_images: false

Run

$ curl -s http://localhost:4000/v1/responses -H "Authorization: Bearer $PROOF_KEY" -H 'Content-Type: application/json' -d '{"model": "bedrock-haiku-4-5", "max_output_tokens": 16, "guardrails": ["no-images"], "input": [{"role": "user", "content": [{"type": "input_text", "text": "What color is this image? Answer with one word"}, {"type": "input_image", "image_url": "data:image/png;base64,$RED"}]}]}' | jq -c 'if .error then {error: .error.message} else {answer: .output[0].content[0].text} end'

Observe
```
{"answer":"Red"}
```
Run
```
$ jq -c . endpoint.jsonl
```

Observe

{"input_type":"request","fields_received":["additional_provider_specific_params","input_type","litellm_call_id","litellm_trace_id","litellm_version","model","request_data","request_headers","structured_messages","texts"],"images":0,"image_urls_in_messages":["data:image/png;base64,iVBORw0KGgoAAAANSU"]}

Case 3: /v1/messages with an image block and send_images: false

Run

$ curl -s http://localhost:4000/v1/messages -H "Authorization: Bearer $PROOF_KEY" -H 'Content-Type: application/json' -d '{"model": "bedrock-haiku-4-5", "max_tokens": 10, "guardrails": ["no-images"], "messages": [{"role": "user", "content": [{"type": "text", "text": "What color is this image? Answer with one word"}, {"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": "$RED"}}]}]}' | jq -c 'if .error then {error: .error.message} else {answer: .content[0].text} end'

Observe
```
{"answer":"Red"}
```
Run
```
$ jq -c . endpoint.jsonl
```

Observe

{"input_type":"request","fields_received":["additional_provider_specific_params","images","input_type","litellm_call_id","litellm_trace_id","litellm_version","model","request_data","request_headers","structured_messages","texts"],"images":1,"image_urls_in_messages":["data:image/png;base64,iVBORw0KGgoAAAANSU"]}

Case 4: exclude_payload_fields: [request_headers, tools]

Run

$ curl -s http://localhost:4000/v1/chat/completions -H "Authorization: Bearer $PROOF_KEY" -H 'Content-Type: application/json' -d '{"model": "bedrock-haiku-4-5", "max_tokens": 10, "guardrails": ["no-headers"], "tools": [{"type": "function", "function": {"name": "lookup", "parameters": {"type": "object", "properties": {}}}}], "messages": [{"role": "user", "content": "Reply with one word: hi"}]}' | jq -c 'if .error then {error: .error.message} else {answer: .choices[0].message.content} end'

Observe
```
{"answer":"Hi!"}
```
Run
```
$ jq -c . endpoint.jsonl
```

Observe

{"input_type":"request","fields_received":["additional_provider_specific_params","input_type","litellm_call_id","litellm_trace_id","litellm_version","model","request_data","request_headers","structured_messages","texts","tools"],"images":0,"image_urls_in_messages":[]}

Case 5: the guardrail rewrites texts it was not sent

Run

$ curl -s http://localhost:4000/v1/chat/completions -H "Authorization: Bearer $PROOF_KEY" -H 'Content-Type: application/json' -d '{"model": "bedrock-haiku-4-5", "max_tokens": 10, "guardrails": ["no-texts"], "messages": [{"role": "user", "content": "my ssn is 123-45-6789, reply with one word: hi"}]}' | jq -c 'if .error then {error: .error.message} else {answer: .choices[0].message.content} end'

Observe

{"answer":"I can't help with that request. Please don"}

Run
```
$ jq -c . endpoint.jsonl
```

Observe

{"input_type":"request","fields_received":["additional_provider_specific_params","input_type","litellm_call_id","litellm_trace_id","litellm_version","model","request_data","request_headers","structured_messages","texts"],"images":0,"image_urls_in_messages":[]}

Case 6: a post_call guardrail masks response texts it was not sent and also returns tools

Run

$ curl -s http://localhost:4000/v1/chat/completions -H "Authorization: Bearer $PROOF_KEY" -H 'Content-Type: application/json' -d '{"model": "bedrock-haiku-4-5", "max_tokens": 20, "guardrails": ["no-texts-response"], "messages": [{"role": "user", "content": "Repeat exactly this line and nothing else: call me at 555-0100"}]}' | jq -c 'if .error then {error: .error.message} else {answer: .choices[0].message.content} end'

Observe
```
{"answer":"call me at 555-0100"}
```
Run
```
$ jq -c . endpoint.jsonl
```

Observe

{"input_type":"response","fields_received":["additional_provider_specific_params","input_type","litellm_call_id","litellm_trace_id","litellm_version","model","request_data","request_headers","texts"],"images":0,"image_urls_in_messages":[]}

Case 7: send_images: "maybe" is ignored with a warning and images are still sent

Run

$ curl -s http://localhost:4000/v1/chat/completions -H "Authorization: Bearer $PROOF_KEY" -H 'Content-Type: application/json' -d '{"model": "bedrock-haiku-4-5", "max_tokens": 10, "guardrails": ["typo"], "messages": [{"role": "user", "content": [{"type": "text", "text": "What color is this image? Answer with one word"}, {"type": "image_url", "image_url": {"url": "data:image/png;base64,$RED"}}]}]}' | jq -c 'if .error then {error: .error.message} else {answer: .choices[0].message.content} end'

Observe
```
{"answer":"Red"}
```
Run
```
$ jq -c . endpoint.jsonl
```

Observe

{"input_type":"request","fields_received":["additional_provider_specific_params","images","input_type","litellm_call_id","litellm_trace_id","litellm_version","model","request_data","request_headers","structured_messages","texts"],"images":1,"image_urls_in_messages":["data:image/png;base64,iVBORw0KGgoAAAANSU"]}

Run

$ grep -o 'Ignoring send_images=.*' litellm.log

Observe: no output

Case 8: startup warning for a guardrail that is not sent everything

Run

$ grep -o 'Generic Guardrail API (no-images): .*' litellm.log

Observe: no output

After (`d761026`)

Case 1: /v1/chat/completions with an image and send_images: false

Run

$ curl -s http://localhost:4000/v1/chat/completions -H "Authorization: Bearer $PROOF_KEY" -H 'Content-Type: application/json' -d '{"model": "bedrock-haiku-4-5", "max_tokens": 10, "guardrails": ["no-images"], "messages": [{"role": "user", "content": [{"type": "text", "text": "What color is this image? Answer with one word"}, {"type": "image_url", "image_url": {"url": "data:image/png;base64,$RED"}}]}]}' | jq -c 'if .error then {error: .error.message} else {answer: .choices[0].message.content} end'

Observe
```
{"answer":"Red"}
```
Run
```
$ jq -c . endpoint.jsonl
```

Observe

{"input_type":"request","fields_received":["additional_provider_specific_params","input_type","litellm_call_id","litellm_trace_id","litellm_version","model","request_data","request_headers","structured_messages","texts"],"images":0,"image_urls_in_messages":["[omitted]"]}

Case 2: /v1/responses with an input_image and send_images: false

Run

$ curl -s http://localhost:4000/v1/responses -H "Authorization: Bearer $PROOF_KEY" -H 'Content-Type: application/json' -d '{"model": "bedrock-haiku-4-5", "max_output_tokens": 16, "guardrails": ["no-images"], "input": [{"role": "user", "content": [{"type": "input_text", "text": "What color is this image? Answer with one word"}, {"type": "input_image", "image_url": "data:image/png;base64,$RED"}]}]}' | jq -c 'if .error then {error: .error.message} else {answer: .output[0].content[0].text} end'

Observe
```
{"answer":"Red"}
```
Run
```
$ jq -c . endpoint.jsonl
```

Observe

{"input_type":"request","fields_received":["additional_provider_specific_params","input_type","litellm_call_id","litellm_trace_id","litellm_version","model","request_data","request_headers","structured_messages","texts"],"images":0,"image_urls_in_messages":["[omitted]"]}

Case 3: /v1/messages with an image block and send_images: false

Run

$ curl -s http://localhost:4000/v1/messages -H "Authorization: Bearer $PROOF_KEY" -H 'Content-Type: application/json' -d '{"model": "bedrock-haiku-4-5", "max_tokens": 10, "guardrails": ["no-images"], "messages": [{"role": "user", "content": [{"type": "text", "text": "What color is this image? Answer with one word"}, {"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": "$RED"}}]}]}' | jq -c 'if .error then {error: .error.message} else {answer: .content[0].text} end'

Observe
```
{"answer":"Red"}
```
Run
```
$ jq -c . endpoint.jsonl
```

Observe

{"input_type":"request","fields_received":["additional_provider_specific_params","input_type","litellm_call_id","litellm_trace_id","litellm_version","model","request_data","request_headers","structured_messages","texts"],"images":0,"image_urls_in_messages":["[omitted]"]}

Case 4: exclude_payload_fields: [request_headers, tools]

Run

$ curl -s http://localhost:4000/v1/chat/completions -H "Authorization: Bearer $PROOF_KEY" -H 'Content-Type: application/json' -d '{"model": "bedrock-haiku-4-5", "max_tokens": 10, "guardrails": ["no-headers"], "tools": [{"type": "function", "function": {"name": "lookup", "parameters": {"type": "object", "properties": {}}}}], "messages": [{"role": "user", "content": "Reply with one word: hi"}]}' | jq -c 'if .error then {error: .error.message} else {answer: .choices[0].message.content} end'

Observe
```
{"answer":"Hi"}
```
Run
```
$ jq -c . endpoint.jsonl
```

Observe

{"input_type":"request","fields_received":["additional_provider_specific_params","input_type","litellm_call_id","litellm_trace_id","litellm_version","model","request_data","structured_messages","texts"],"images":0,"image_urls_in_messages":[]}

Case 5: the guardrail rewrites texts it was not sent

Run

$ curl -s http://localhost:4000/v1/chat/completions -H "Authorization: Bearer $PROOF_KEY" -H 'Content-Type: application/json' -d '{"model": "bedrock-haiku-4-5", "max_tokens": 10, "guardrails": ["no-texts"], "messages": [{"role": "user", "content": "my ssn is 123-45-6789, reply with one word: hi"}]}' | jq -c 'if .error then {error: .error.message} else {answer: .choices[0].message.content} end'

Observe

{"error":"Guardrail 'no-texts' rewrote the request in a way this endpoint cannot apply, so the request was rejected rather than sent unrewritten"}

Run
```
$ jq -c . endpoint.jsonl
```

Observe

{"input_type":"request","fields_received":["additional_provider_specific_params","input_type","litellm_call_id","litellm_trace_id","litellm_version","model","request_data","request_headers","structured_messages"],"images":0,"image_urls_in_messages":[]}

Case 6: a post_call guardrail masks response texts it was not sent and also returns tools

Run

$ curl -s http://localhost:4000/v1/chat/completions -H "Authorization: Bearer $PROOF_KEY" -H 'Content-Type: application/json' -d '{"model": "bedrock-haiku-4-5", "max_tokens": 20, "guardrails": ["no-texts-response"], "messages": [{"role": "user", "content": "Repeat exactly this line and nothing else: call me at 555-0100"}]}' | jq -c 'if .error then {error: .error.message} else {answer: .choices[0].message.content} end'

Observe

{"error":"Guardrail 'no-texts-response' rewrote the response in a way this endpoint cannot apply, so the response was rejected rather than sent unrewritten"}

Run
```
$ jq -c . endpoint.jsonl
```

Observe

{"input_type":"response","fields_received":["additional_provider_specific_params","input_type","litellm_call_id","litellm_trace_id","litellm_version","model","request_data","request_headers"],"images":0,"image_urls_in_messages":[]}

Case 7: send_images: "maybe" is ignored with a warning and images are still sent

Run

$ curl -s http://localhost:4000/v1/chat/completions -H "Authorization: Bearer $PROOF_KEY" -H 'Content-Type: application/json' -d '{"model": "bedrock-haiku-4-5", "max_tokens": 10, "guardrails": ["typo"], "messages": [{"role": "user", "content": [{"type": "text", "text": "What color is this image? Answer with one word"}, {"type": "image_url", "image_url": {"url": "data:image/png;base64,$RED"}}]}]}' | jq -c 'if .error then {error: .error.message} else {answer: .choices[0].message.content} end'

Observe
```
{"answer":"Red"}
```
Run
```
$ jq -c . endpoint.jsonl
```

Observe

{"input_type":"request","fields_received":["additional_provider_specific_params","images","input_type","litellm_call_id","litellm_trace_id","litellm_version","model","request_data","request_headers","structured_messages","texts"],"images":1,"image_urls_in_messages":["data:image/png;base64,iVBORw0KGgoAAAANSU"]}

Run

$ grep -o 'Ignoring send_images=.*' litellm.log

Observe

Ignoring send_images='maybe', expected a bool. Images are sent

Case 8: startup warning for a guardrail that is not sent everything

Run

$ grep -o 'Generic Guardrail API (no-images): .*' litellm.log

Observe

Generic Guardrail API (no-images): ['images'] are not sent to the guardrail, so it can only enforce on what it is sent, and it cannot rewrite what it did not see.

Type

🆕 New Feature

Caveats (if any)

Severe

Withheld or excluded content is never scanned, so it cannot be blocked or masked
- With send_images: false, text written inside an image is not checked
- Startup logs a warning naming the fields that are not sent
- input_type and litellm_call_id are always sent

Medium

A rewrite that cannot be applied is rejected with HTTP 500
- The proxy also logs it as "looks like a bug in LiteLLM"
- The error type has no status code yet, a follow-up outside this PR
- It happens when the guardrail changes, moves or copies an [omitted] part
- It also happens when its only real change went to a field it was not sent
- On the response side the paid model call has already happened
send_images: false only covers image parts
- Remote image URLs and /v1/messages PDFs are withheld too
- File, audio and video parts, including image data inside file, are still sent

Low

exclude_payload_fields is config-only, the Admin UI form hides it
- The form has no list input yet, the same ui_hidden marker as feat(guardrails): add system prompt and first role skip filters to generic_guardrail_api #43684 does this
An invalid value for either option is ignored with a warning and its default is kept
- Unknown field names in exclude_payload_fields are also ignored with a warning
Tools and rows count as an applied rewrite only when the endpoint posted them
- So a tool the guardrail adds to a request that posted none cannot stand in for a refused rewrite, and that call fails closed
A2A requests never reach this guardrail, here or on main
- The A2A message has parts and no content, so the guardrail request fails validation and the call fails closed
- So a masked row the A2A handler would ignore can never let a message through, a separate bug from this PR
The guardrail is not told which fields were withheld, it only sees them missing
image_part_url overlaps the private _image_part_url in prompt_templates/common_utils.py
- That one only takes mappings, so reusing it means widening a shared helper outside this PR
Merge conflicts are expected with the sibling PRs split out of feat(guardrails): add payload, applicability, dispatch and logging controls to generic_guardrail_api #37055
- config_parsing.py is also added by feat(guardrails): add system prompt and first role skip filters to generic_guardrail_api #43684, feat(guardrails): add key alias and team skip filters to generic_guardrail_api #43757 and feat(guardrails): add call-type filters to generic_guardrail_api #43765, keep this copy's fallback argument
- "Keep both" conflicts in generic_guardrail_api.py with feat(guardrails): add guardrail_information_scope to generic_guardrail_api #43680, feat(guardrails): add fire_and_forget dispatch to generic_guardrail_api #43681, feat(guardrails): add system prompt and first role skip filters to generic_guardrail_api #43684, feat(guardrails): add key alias and team skip filters to generic_guardrail_api #43757 and feat(guardrails): add call-type filters to generic_guardrail_api #43765
- feat(guardrails): add system prompt and first role skip filters to generic_guardrail_api #43684 also edits _content_utils.py
- feat(guardrails): add max_messages, max_text_chars and strip_patterns to generic_guardrail_api #43761 is stacked on an older version of this branch and needs a rebase

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

CLAassistant · 2026-09-29T19:30:43Z

All committers have signed the CLA.

codecov · 2026-09-29T19:34:00Z

Codecov Report

❌ Patch coverage is 98.49624% with 4 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/guardrails/_content_utils.py	93.02%	3 Missing ⚠️
...rail_hooks/generic_guardrail_api/payload_policy.py	99.33%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

codspeed · 2026-09-29T19:42:17Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing caduri:feature/generic-guardrail-payload-fields (d761026) with main (254c2f6)}

The request model validates list message content lazily, so the dump that builds the POST body consumes it and a later read of the model rows sees empty content. apply_guardrail compared the guardrail's returned rows with those consumed rows, so no multipart row ever matched its echo. A guardrail that echoed every row unchanged and masked through texts had its rows taken as a rewrite, and the unmasked rows reached the LLM Compare against the JSON rows actually posted instead. An unchanged echo now falls back to texts again, and a partial echo restores the caller's row at each echoed index. structured_messages_from_response is renamed to structured_messages_from_json since it now reads request rows as well The dump also sends a row as content [] when one of its parts fails validation (guarded_text, a base64 document, flac audio), so the guardrail never saw that row's text. An echo of it restored the caller's unmasked row while another row's rewrite made the rows path win, which dropped the masked texts. Such a row now carries the caller's own content, so the guardrail sees it in full. Rows the model dumps in full are posted exactly as before, and the rest of the body is unchanged That content goes through a new as_json_value helper in _content_utils. It round-trips through the stdlib json codec because pydantic's serializer silently replaces anything nested past 254 levels with "..." GenericGuardrailAPI also takes an optional async_handler so tests can inject the HTTP client. The regression tests drive the real OpenAI chat and Anthropic Messages guardrail handlers through it

A row whose part list holds a part the request model rejects is now posted with the caller's content as is. The as_json_value round trip it went through is gone: the whole request is already dumped by pydantic before that point, so it never kept extra nesting depth, and no test needed it The two JSON builds in the row helpers carry the same mutable-ok as the POST body, and a new test covers /v1/responses input_text rows, which the fix already handled

A rejected part holding a non-JSON-native value, such as a tuple, was posted as a JSON array but compared against the caller's tuple. An unchanged echo then looked rewritten, so the masked texts were dropped and the original prompt reached the LLM Caller content now goes through the same stdlib codec httpx encodes the body with before it is posted and compared, and a regression test covers the tuple case

…drail_api Main dropped the LIT002 rule, so the mutable-ok comments on the JSON row builds no longer suppress anything and LIT013 now flags them. They are removed. The part-list guard narrows to Sequence[object] instead of list[object], since its callers only need len() and a read, which also keeps it clear of LIT001

        yield from _iter_text_parts_in_content(message.get("content"))


+def image_part_url(part: object) -> str | None:


caduri · 2026-10-04T14:03:18Z

@greptileai please review the latest tip d11066e, which is rebased onto #43679 and adds the live proof and config-only fields

greptile-apps · 2026-10-04T14:08:46Z

Confidence Score: 5/5

[Medium risk] Adds image and payload filtering to the generic guardrail API.

The PR appears safe to merge; the remaining concern is non-blocking latency and memory cost when large structured rows are echoed.

Findings

Repeated row serialization ▶

Summary

The PR adds options to withhold images and selected payload fields from the Generic Guardrail API, restores withheld images on applicable rewrites, and rejects interventions it cannot apply.

The latest revision centralizes the sentinels and treats echoed rows without null-valued fields as unchanged.
Echo comparison introduces avoidable serialization cost for large image-bearing rows.

_{Reviews (3) · Last reviewed commit: "refactor(guardrails): move the payload s..."}

caduri · 2026-10-04T14:58:54Z

@greptileai please re-review 3465bfb, which fixes the response rewrite P1 by counting tools and rows only when the endpoint posted them

A guardrail that re-serializes the rows it was sent often drops null fields, such as the thinking_blocks: null an Anthropic assistant row is posted with or the content: null of a chat tool-call row. The echo check compared rows exactly, so such an echo looked like a rewrite of every row: the masked texts were ignored and the unmasked rows written back, and the raw value reached the model. Rows now count as an echo when they match apart from null fields, in both the every-row check and the per-row restore. A field the guardrail sets to null where the caller had a value still counts as a change

…ic_guardrail_api send_images=False drops the top-level images field and replaces the URL of every image_url part in the structured_messages rows sent to the guardrail with "[omitted]", keeping the part so the guardrail still knows an image was there. That covers remote URLs as well as inline data, and on /v1/messages Anthropic documents too, since they reach the guardrail as image parts. exclude_payload_fields drops named top-level request fields. Unknown names warn and are ignored, and input_type and litellm_call_id are always sent. An invalid value for either option is ignored with a warning and its default is kept The guardrail can no longer rewrite content it did not see. Returned texts, structured_messages, images and tools are ignored with a warning when that field was not sent. An intervention whose only real change went to such a field is rejected instead of letting the call through unchanged, and the error names the request or response side. Changes are compared with what the guardrail was posted, and only texts, tools and written-back rows count, since no endpoint writes returned images back. A returned row whose image was withheld is accepted with the caller's image put back into each part still holding the placeholder, and a rewrite that changes, moves or copies a placeholder is rejected exclude_payload_fields is config-only: fields marked ui_hidden are left out of the Admin UI form, which has no list input for it. initialize_guardrail accepts an optional async_handler, so tests inject the HTTP client

…nt posted them A guardrail that was not sent texts could mask them and also return tools or structured_messages. The refusal check counted those as an applied change, but every response handler and most request handlers (embeddings, completion, image generation and others) never supply or apply either field, so the call went through with the original, unmasked text Tools and written-back rows now count only when the endpoint posted them to the guardrail, which covers both sides without a per-endpoint list. A tool the guardrail adds to a request that posted none no longer stands in for a refused rewrite either, so that case now fails closed

The repo keeps sentinel values in litellm/constants.py, so the "[omitted]" image placeholder and the marker for a rewrite that cannot be applied move there as GENERIC_GUARDRAIL_IMAGE_OMITTED_PLACEHOLDER and GENERIC_GUARDRAIL_UNAPPLIABLE_REWRITE. The marker stays a frozen dataclass so the restore helpers can still narrow their unions with isinstance It also adds a regression test for a guardrail that is not sent texts, masks them anyway and echoes Anthropic rows without their null fields. With the null fields now ignored in the echo check, nothing applies and the request is rejected instead of reaching the model unmasked

caduri · 2026-10-05T09:19:59Z

@greptileai please review d761026, rebased onto #43679's null-echo fix with the payload sentinels moved into constants.py

greptile-apps · 2026-10-05T09:23:47Z

    if original_rows is None or shown_rows is None or len(returned_rows) != len(original_rows):
        return tuple(returned_rows)
-    if all(returned == shown for shown, returned in zip(shown_rows, returned_rows)):
+    if all(same_json_ignoring_nulls(returned, shown) for shown, returned in zip(shown_rows, returned_rows)):


Repeated row serialization When a guardrail echoes structured messages containing large base64 images, this comparison serializes and parses both the returned and posted rows. If any row differs, the code repeats that work for each row. This adds latency and raises peak memory use on image-bearing requests.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Measured about 0.06s per call on a 20 MB base64 row, the same order as the request's existing JSON serialization, so leaving it as is

That measurement addresses the concern: ~0.06 s for a 20 MB row is comparable to the request’s existing JSON serialization, so the additional serialization does not appear to introduce a material regression. Leaving this implementation as-is is reasonable; no change needed.

github-advanced-security Bot found potential problems Sep 29, 2026

View reviewed changes

Comment thread litellm/proxy/guardrails/guardrail_hooks/generic_guardrail_api/payload_policy.py Fixed

caduri mentioned this pull request Sep 29, 2026

feat(guardrails): add max_messages, max_text_chars and strip_patterns to generic_guardrail_api #43761

Draft

5 of 6 tasks

caduri added 4 commits October 4, 2026 15:01

caduri force-pushed the feature/generic-guardrail-payload-fields branch 2 times, most recently from 117c37d to d11066e Compare October 4, 2026 13:50

github-advanced-security Bot found potential problems Oct 4, 2026

View reviewed changes

Comment thread litellm/proxy/guardrails/_content_utils.py

yield from _iter_text_parts_in_content(message.get("content"))

def image_part_url(part: object) -> str | None:

caduri mentioned this pull request Oct 4, 2026

docs(generic_guardrail_api): document send_images and exclude_payload_fields BerriAI/litellm-docs#2075

Draft

greptile-apps Bot reviewed Oct 4, 2026

View reviewed changes

Comment thread litellm/proxy/guardrails/guardrail_hooks/generic_guardrail_api/payload_policy.py

caduri added 4 commits October 5, 2026 12:02

caduri force-pushed the feature/generic-guardrail-payload-fields branch from 3465bfb to d761026 Compare October 5, 2026 09:17

greptile-apps Bot reviewed Oct 5, 2026

View reviewed changes

caduri mentioned this pull request Oct 5, 2026

feat(guardrails): add call-type filters to generic_guardrail_api #43765

Draft

6 tasks done

Navigation Menu

Repository navigation

feat(guardrails): add send_images and exclude_payload_fields to generic_guardrail_api - #43755

feat(guardrails): add send_images and exclude_payload_fields to generic_guardrail_api#43755
caduri wants to merge 8 commits into
BerriAI:mainfrom
caduri:feature/generic-guardrail-payload-fields

caduri commented Sep 29, 2026 •

edited

Loading

Uh oh!

CLAassistant commented Sep 29, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Sep 29, 2026 •

edited

Loading

Uh oh!

Uh oh!

codspeed Bot commented Sep 29, 2026 •

edited

Loading

Uh oh!

caduri commented Oct 4, 2026

Uh oh!

greptile-apps Bot commented Oct 4, 2026 •

edited

Loading

Uh oh!

Uh oh!

caduri commented Oct 4, 2026

Uh oh!

caduri commented Oct 5, 2026

Uh oh!

greptile-apps Bot Oct 5, 2026

Uh oh!

caduri Oct 5, 2026

Uh oh!

greptile-apps Bot Oct 5, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		yield from _iter_text_parts_in_content(message.get("content"))


		def image_part_url(part: object) -> str \| None:

Conversation

caduri commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

Relevant issues

User Flow

Pre-Submission checklist

Delays in PR merge?

Screenshots / Proof of Fix

Before (5b61f8b)

Case 1: /v1/chat/completions with an image and send_images: false

Case 2: /v1/responses with an input_image and send_images: false

Case 3: /v1/messages with an image block and send_images: false

Case 4: exclude_payload_fields: [request_headers, tools]

Case 5: the guardrail rewrites texts it was not sent

Case 6: a post_call guardrail masks response texts it was not sent and also returns tools

Case 7: send_images: "maybe" is ignored with a warning and images are still sent

Case 8: startup warning for a guardrail that is not sent everything

After (d761026)

Case 1: /v1/chat/completions with an image and send_images: false

Case 2: /v1/responses with an input_image and send_images: false

Case 3: /v1/messages with an image block and send_images: false

Case 4: exclude_payload_fields: [request_headers, tools]

Case 5: the guardrail rewrites texts it was not sent

Case 6: a post_call guardrail masks response texts it was not sent and also returns tools

Case 7: send_images: "maybe" is ignored with a warning and images are still sent

Case 8: startup warning for a guardrail that is not sent everything

Type

Caveats (if any)

Severe

Medium

Low

Final Attestation

Uh oh!

CLAassistant commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

codspeed Bot commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Uh oh!

caduri commented Oct 4, 2026

Uh oh!

greptile-apps Bot commented Oct 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Findings

Summary

Uh oh!

Uh oh!

caduri commented Oct 4, 2026

Uh oh!

caduri commented Oct 5, 2026

Uh oh!

greptile-apps Bot Oct 5, 2026

Choose a reason for hiding this comment

Uh oh!

caduri Oct 5, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Oct 5, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

caduri commented Sep 29, 2026 •

edited

Loading

Before (`5b61f8b`)

After (`d761026`)

CLAassistant commented Sep 29, 2026 •

edited

Loading

codecov Bot commented Sep 29, 2026 •

edited

Loading

codspeed Bot commented Sep 29, 2026 •

edited

Loading

greptile-apps Bot commented Oct 4, 2026 •

edited

Loading