feat(guardrails): add llm shield pii redaction and rehydration guardrail by ninadphalak · Pull Request #42645 · BerriAI/litellm

ninadphalak · 2026-09-23T01:52:27Z

Docs and end-to-end check (Oct 3)

Docs: BerriAI/litellm-docs#2070 adds docs/proxy/guardrails/llm_shield_proxy.md and its sidebar entry.

How to verify it end to end. LLM Shield is self-hosted, so there is no vendor API key to hand over; the whole loop runs locally in Docker. Three containers: the public LLM Shield image, a small stand-in OpenAI upstream that echoes the user's message back (streamed in 3-character chunks, so a restored value has to be rebuilt across chunk boundaries), and ghcr.io/berriai/litellm:main-latest with this PR's litellm/ package mounted over the image's. Save the three files below in one folder, then:

LITELLM_SRC=/path/to/this/pr/checkout docker compose up -d

curl -s localhost:4000/v1/chat/completions -H "Authorization: Bearer sk-litellm-test" -H "Content-Type: application/json" \
  -d '{"model":"echo","messages":[{"role":"user","content":"Email jane.doe@example.com the invoice for card 4111 1111 1111 1111"}]}'
# same request with "stream": true, then:
docker compose logs echo
docker compose stop shield   # the next request is blocked with a 400, and the upstream receives nothing

What I get on 1b45935b: the caller receives Email jane.doe@example.com the invoice for card 4111 1111 1111 1111 both streaming and non-streaming, while the upstream log shows it only ever saw stand-ins (upstream received: 'Email john10@example.net the invoice for card 6567211639751374'). With the Shield stopped, the request fails with LLM Shield Proxy is unreachable; blocking the request. and the upstream request count does not move.

compose.yaml

services:
  shield:
    image: ghcr.io/ninadphalak/llm-shield-proxy:latest
    environment:
      VALID_VIRTUAL_KEYS: sk-shield-test
  echo:
    image: python:3.12-slim
    command: ["python", "/echo_upstream.py"]
    volumes: ["./echo_upstream.py:/echo_upstream.py:ro"]
  litellm:
    image: ghcr.io/berriai/litellm:main-latest
    command: ["--config", "/config.yaml", "--port", "4000"]
    environment:
      LITELLM_MASTER_KEY: sk-litellm-test
    volumes:
      - ./config.yaml:/config.yaml:ro
      # This PR's checkout, laid over the image's package.
      - ${LITELLM_SRC:?set LITELLM_SRC to a checkout of the PR branch}/litellm:/app/.venv/lib/python3.13/site-packages/litellm:ro
    ports: ["4000:4000"]
    depends_on: [shield, echo]

config.yaml

model_list:
  - model_name: echo
    litellm_params:
      model: openai/echo
      api_base: http://echo:8765/v1
      api_key: sk-not-used

guardrails:
  - guardrail_name: llm-shield
    litellm_params:
      guardrail: llm_shield_proxy
      mode: ["pre_call", "post_call"]
      default_on: true
      api_base: http://shield:8000
      api_key: sk-shield-test

echo_upstream.py

"""A stand-in OpenAI upstream: replies with the last user message, and prints what it received.

Streaming replies are cut into 3-character chunks, so a restored value has to be put back
together across chunk boundaries.
"""

import json
import time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer


class Echo(BaseHTTPRequestHandler):
    def do_POST(self):
        body = json.loads(self.rfile.read(int(self.headers["Content-Length"])))
        text = next(m["content"] for m in reversed(body["messages"]) if m["role"] == "user")
        print(f"upstream received: {text!r}", flush=True)
        base = {"id": "chatcmpl-echo", "created": int(time.time()), "model": body["model"]}
        if not body.get("stream"):
            reply = {
                **base,
                "object": "chat.completion",
                "choices": [{"index": 0, "message": {"role": "assistant", "content": text}, "finish_reason": "stop"}],
                "usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2},
            }
            payload = json.dumps(reply).encode()
            self.send_response(200)
            self.send_header("Content-Type", "application/json")
            self.send_header("Content-Length", str(len(payload)))
            self.end_headers()
            self.wfile.write(payload)
            return
        self.send_response(200)
        self.send_header("Content-Type", "text/event-stream")
        self.end_headers()
        pieces = [text[i : i + 3] for i in range(0, len(text), 3)]
        for index, piece in enumerate(pieces):
            last = index == len(pieces) - 1
            chunk = {
                **base,
                "object": "chat.completion.chunk",
                "choices": [{"index": 0, "delta": {"content": piece}, "finish_reason": "stop" if last else None}],
            }
            self.wfile.write(f"data: {json.dumps(chunk)}\n\n".encode())
        self.wfile.write(b"data: [DONE]\n\n")

    def log_message(self, *args):
        pass


ThreadingHTTPServer(("0.0.0.0", 8765), Echo).serve_forever()

If you would rather keep this under test in CI, the same three pieces fit an integration job, and I am happy to add one in whatever form you use for other guardrails.

lint went red on Oct 2 after a merge of main brought in LIT013, LIT014 and a now-exhausted reportUnknownArgumentType budget; 6db02606 and 1b45935b fix it, and every check is green again.

Update — Sep 26

The review's open points are addressed in f1a68905, 40c26178, d7d608fc, 1b81bb45, 0f9ca3fd and f1578d62:

Native streams are restored. /v1/messages and /v1/responses streams have no
choices, so they used to pass through still holding placeholders. Both now restore
incrementally with the same per-stream sliding windows as chat:
- /v1/messages (raw SSE): frames are cut at event boundaries, so an event split
  across network chunks is parsed once it is whole. text_delta and
  input_json_delta are restored per block index, and held text goes out as one more
  delta just ahead of content_block_stop. Signed thinking_delta blocks, frames from
  other endpoints and raw streams that are not SSE pass through unchanged.
- /v1/responses (typed events): matched by shape, not by a list of families. Any
  *.delta with a string delta gets its own window, so MCP arguments, reasoning
  summaries and families added later are covered too. Held text goes out as a copy of
  the stream's last delta before its *.done event. Every *.done event, and
  response.completed, is then restored in full.
More request fields are redacted. Anthropic tool_use.input and Responses
reasoning summary parts, into the caller's vault. Tool and legacy function
descriptions (including every description in a parameter schema), user,
safety_identifier and web-search user_location city/region go to the
non-restorable vault that system prompts already use.
Structured-output schemas (response_format.json_schema, Responses text.format) get
the same treatment, and prediction.content is redacted into the caller's vault.
Names, types and enum values are left alone, since the model has to reproduce them.
Request walks fail closed. Text nested past a walk's depth bound used to be
skipped, which forwarded it unredacted; an existing test asserted exactly that. Such a
request is now refused. The bounds sit far beyond real payloads.
Tool and output schemas are scanned by default. Every string is redacted into the
non-restorable vault except under keywords whose value must go out verbatim (types,
formats, patterns, $refs, required lists). Under properties, keys are property
names, so a property called type is still walked. enum / const values go to
the caller's vault instead: the model emits the stand-in, and restoring the reply
turns it back into the value the schema allows.
Tool results deliberately stay restorable (reasoning in the thread on that line).

Verified beyond the unit tests: a script ran the hook against the real LLM Shield guard
API, with a real email redacted and the stand-in echoed back split at every one of 50
character offsets, for both stream shapes. The client always received the original,
never a fragment.

lint was also red since Sep 23 (a LIT002 budget overrun, then 11 Final-in-loop
basedpyright errors that the first failure had been hiding). Both are fixed in
b9656734 and 12e3b14a. 92 tests pass.

Issue Number

Reopens #39621.

That PR was closed on Sep 13 when its base branch, litellm_internal_staging, was
deleted — GitHub auto-closes any PR pointing at a removed base. Nothing was requested
and nothing is in dispute; this is the same work retargeted at main.

CLA is signed.

Correcting something I wrote when I opened this. I cited #39621's Greptile 5/5 as
the state it closed in. That was wrong, and worth saying before a reviewer spends time
on it: the 5/5 was scored on Sep 3, and the branch took four more commits after it. The
last of those introduced a call to copy.deepcopy with no import copy — a guaranteed
NameError on any response carrying tool calls. Greptile never re-reviewed, so the
score stood over code that had a crash in it.

Greptile found it immediately on this PR and scored it 1/5. It and the two other real
findings are fixed in 28697377:

the missing import copy, plus a regression test on the response path (every existing
tool-call test covered the request side, which is why 47 tests passed over a certain
NameError);
the vault session id moved from metadata to litellm_metadata. veria-ai was right
that this matters: metadata is forwarded upstream on /v1/responses, so a provider
holding both the placeholders and the session id could call the shield's rehydrate
endpoint and recover the plaintext this guardrail exists to withhold;
_collect_json_leaves rewritten with an explicit stack, since it recursed over
model-controlled JSON and the recursive_detector gate rejects that.

52 tests pass. ruff format, ruff-strict and check_type_discipline are clean, with
LIT counts identical to the merge base.

Summary

Adds an optional LLM Shield Proxy guardrail. It redacts PII from outbound request text
before it reaches a provider, and rehydrates the original values on the way back, so the
caller sees real data while the provider never does.

The rehydration half is the part that does not already exist in tree: existing PII
guardrails here are one-way: once a value is masked, the response carries the mask. This
restores the original values on non-streaming responses and on streaming deltas, keyed
to a session-scoped vault.

Changes

Guardrail hook at litellm/proxy/guardrails/guardrail_hooks/llm_shield_proxy/,
registered through the standard guardrail registry and config model.
Request traversal covering chat, Responses, Completions, Anthropic and tool-call
request shapes.
Session-scoped rehydration for regular responses, and incremental restoration for
streaming ones: chat per choice, Anthropic /v1/messages SSE per block, and
/v1/responses events per part.
Fail-closed transport: if the Shield endpoint is unreachable, the request is refused
rather than forwarded unredacted.
Dashboard discovery, configuration presets and branding.
Tests covering request shapes, lifecycle, failure modes and streaming restoration
for all three stream shapes.

The diff is a thin HTTP client. Detection and the rehydration vault live in the external
service, so nothing heavyweight is vendored into this tree.

Notes for review

Base is now main rather than litellm_internal_staging; say the word if you would
rather it target something else.
Merged current main in. The only conflicts were five registration points where
upstream added the Agent365 guardrail at the same position this branch adds LLM
Shield — the enum, the garden config and card lists, the logo map and its parity
test. Both entries are kept in all five.

Note

High Risk
Changes the proxy data path for PII redaction/rehydration, session vault isolation, and response-cache behavior; mis-wiring could leak plaintext or block legitimate traffic.

Overview
Adds a new LLM Shield Proxy guardrail that sends outbound text to a self-hosted Shield service for reversible PII redaction on pre_call, then rehydrates placeholders on post_call so callers see originals while providers only see stand-ins.

The hook walks chat, Completions, Responses, and Anthropic request shapes (including tools, schemas, and extra_body), splits caller-restorable text from privileged server-authored spans in separate session vaults, and blocks the call if Shield is down or nesting exceeds bounds. Response handling restores on copies to keep placeholders in LiteLLM’s cache, with incremental stream restoration for chat deltas, Anthropic SSE, and Responses events; model-level SDK use outside the proxy disables caching and refuses streaming.

Registration adds llm_shield_proxy to guardrail types, config model, example YAML, dashboard garden preset (both pre_call and post_call), logo, and a large unit test suite.

^{Reviewed by Cursor Bugbot for commit db41912. Bugbot is set up for automated code reviews on this repo. Configure here.}

LLM Shield is a self-hosted PII gateway. This adds it as a guardrail so a proxy operator can redact personal data out of outbound requests and have the original values restored in the model's reply. The substitution is reversible, which is the difference from a masking guardrail. Outbound text is replaced with placeholders held in a session vault inside the operator's own LLM Shield deployment, and the reply is restored before it reaches the caller, so the end user still sees real values while the provider never received them. Streaming responses are restored incrementally. LLM Shield holds back only the trailing characters that could still turn out to be part of a placeholder, so tokens are forwarded as they arrive rather than the whole response being collected first. A placeholder split across two chunks is never emitted in fragments. The integration talks to LLM Shield over HTTP and adds no dependency. Notes for reviewers: - The guardrail sets use_native_lifecycle_hooks, since redaction and restoration need the native pre-call, post-call and streaming hooks rather than the unified path. - Per-request state lives on the request dict, never on the guardrail instance, because the proxy registers a single instance process-wide. The streaming carry-over is a local of the generator for the same reason. - Every failure blocks the request. A redaction guardrail that fails open would send the exact data it exists to protect to the provider.

Adds the card, preset and logo so operators can pick LLM Shield from the guardrails page the same way as the other partner guardrails.

Shows both modes on one entry. Listing only pre_call redacts the request and then hands the placeholders back to the end user, so the test asserts both hooks are enabled.

The /v1/messages reply is a plain dict with a content block list and no choices, so it fell through the restore path and went back to the caller still carrying placeholders. The request was redacted correctly, which is what made this easy to miss. Found by running all three endpoints against a live provider; the mocked tests all passed because they only built the OpenAI shape. Adds tests for the message shape and for leaving non-text blocks alone.

…hape Three gaps, all of which let an enabled guardrail hand data to the provider or hand placeholders to the caller. Requests only walked `messages`. The Responses API `input` and tool call `arguments` went out untouched. Measured against a live provider: a request sent through `/v1/responses` reached the model with the real address in it while the guardrail reported as enabled. Request traversal now covers chat content (string and multimodal), tool call arguments, and `input` as a bare string or a list of items. Fixing that exposed the matching gap on the way back: the Responses API reply carries `output` items rather than `choices`, so it returned to the caller still holding placeholders. It now gets its own walk, handling text blocks as dicts or objects. The dashboard preset seeded only pre_call, so a guardrail created from the UI would redact the request and return the placeholders to the user. Presets can now seed both modes; the form already normalised either shape. Adds tests for each request shape, for both Responses API reply forms, and replaces a test that had asserted the `input` bypass as correct behaviour.

basedpyright could not prove the delta was non-None on the write path, and reportOptionalMemberAccess has a zero budget. The guard is also clearer than relying on the text check to imply it.

The vault id was taken from caller-supplied session metadata, and every caller shares one LLM Shield key. Someone who knew or guessed another caller's session id could send a placeholder, have the model echo it back, and get that caller's plaintext restored into their own reply. Vault ids are now minted per request behind a per-process prefix, so a caller cannot name a vault this process uses. Redaction mints, restoration reads back, and a reply whose id does not match is left holding its placeholders rather than resolved against some other vault. Also covers two more request fields that were reaching the provider intact: the Responses API `instructions`, and the legacy `function_call.arguments` alongside `tool_calls`. The collectors move to module level, which drops the traversal back under the complexity limit and lets the code carry its own explanation instead of the comments that were restating it.

basedpyright rejects a Final assigned inside a loop, and reportGeneralTypeIssues sits one over its budget ceiling.

Two more provider-bound request shapes were reaching the model intact while the guardrail reported as enabled. /v1/completions carries its text in a top-level `prompt`, which the traversal never looked at. It is handled as a string and as the array form, where each entry is rewritten in place. Responses input items hold tool data outside `content`: a function_call item in `arguments`, a function_call_output item in `output`. Both are now collected alongside the item's content. Adds a test per shape.

…input Two more provider-bound shapes, found by walking the request types rather than waiting for them to be reported. /v1/messages carries its system prompt at the top level, as a string or a list of text blocks. It is one of the endpoints this guardrail claims to cover, and a system prompt is a natural place to put a customer's details. `input` as an array of bare strings, the embeddings and moderations shape, was skipped because the loop only handled item dicts. Verified against a live provider: a system prompt holding an address now reaches the model as a stand-in and is restored in the reply.

Guarding with a conditional iterable left the value un-narrowed, so passing it on was an argument-type error and the element checks read as unreachable. An early return narrows it properly and reads better.

Streaming rehydration read and rewrote choices[0] only, so with n>1 every later choice went back to the caller still holding its placeholders. Each choice is its own token stream, so the sliding window is now tracked per choice index rather than once per stream. A single shared window would have been worse than the bug: it would splice the characters held back for one choice onto the next one's delta. The final flush walks every choice the same way, and the two helpers that only ever looked at choices[0] are gone. Adds a test that both choices come back restored, and one that each choice gets its own window handed back rather than its neighbour's.

The integration was called llm_shield in code, llm-shield in the example config, and LLM Shield in the dashboard, while the product and its PyPI package are both llm-shield-proxy. An operator who saw the guardrail in LiteLLM could not tell what to install. One identifier now: llm_shield_proxy for the enum value, module, directory, class, config model, logo and environment variables, with LLM Shield Proxy as the display name. That matches `pip install llm-shield-proxy`. Renames only; no behaviour change.

`name` on a user or assistant turn identifies a person and was going to the provider intact. The proxy this integrates with already redacts it, so the integration was the weaker of the two. On a tool or function turn the same field carries the function's name, which has to arrive unchanged or the call stops routing. That case is skipped, and a test asserts the value is never even sent to the shield.

…suffix Three review findings. The trailing flush walked the last chunk's choices, so a choice that finished earlier and stopped appearing lost whatever text was still held for it and its answer was truncated. It is now driven by the windows themselves and emits one chunk per choice, synthesising the choice when the terminal chunk omits it. That was data loss, not just under-redaction. An Anthropic tool_result carries its own content, as a string or as further blocks, and only each part's `text` was being collected. Handled recursively; image and audio parts still fall through untouched. The legacy completions `suffix` is forwarded to providers that support it and was never collected. Note the placement: it has to be gathered before the string-prompt early return, which is what the new test pins.

…ound CI flagged _collect_content as recursive. It was, and worse, it was unbounded: a tool_result nests its own content, the nesting is caller controlled, and the descent had nothing to stop it. That is a JSON bomb, not a style issue. Now an explicit queue with a depth bound of 8. Real payloads nest one or two deep. The queue is walked in document order because the shield maps its replies back by position, so collection order is part of the contract.

A Responses request can send `prompt` as a PromptObject rather than a string. Its `variables` are substituted into the stored prompt on the provider side, so they are caller text, and the dict shape was falling through untouched. `id` and `version` pick which stored prompt to run and are left unchanged.

… end The depth test asserted nothing, so it passed whether or not the bound held, and the test-quality gate counted it as a zero-assert test. It now sends a shallow value alongside a 200-deep chain and asserts the shallow one is collected while the value past the bound is not.

Redaction put every span of a request into one vault, and the reply was restored against that same vault. System prompts are written by the application and the caller never sees them, so a caller who got the model to echo a placeholder back had its plaintext restored into their own reply -- a way to read a system prompt they were never shown. Server-authored spans now go into a vault of their own: system and developer turns, Anthropic's top-level `system`, and the Responses API `instructions`. Its id is deliberately never stored, so nothing restores against it. The reply is restored against the caller's vault alone, and an echoed placeholder from a system prompt comes back as the placeholder. Values the caller also wrote themselves are unaffected -- they are in the caller's vault too, and still restore. The extra round trip happens only when a request actually carries server-authored text.

`ruff format` wanted the widened `_collect_responses_fields` signature on one line, and the three tests added with the split-vault fix needed return annotations to keep ANN201 level with the base.

…rail Both sides appended a new guardrail integration at the same position in the integration enum, the dashboard provider/config/logo maps, the garden card list, and its logo test. Keep both additions: the incoming `conduct` guardrail from staging and this branch's `llm_shield_proxy`. Conflicted files and how they were resolved (both entries retained, in the order ours-then-incoming): litellm/types/guardrails.py ui/litellm-dashboard/src/app/(dashboard)/guardrails/_components/guardrail_garden_configs.ts ui/litellm-dashboard/src/app/(dashboard)/guardrails/_components/guardrail_garden_data.ts ui/litellm-dashboard/src/app/(dashboard)/guardrails/_components/guardrail_garden_data.test.ts ui/litellm-dashboard/src/app/(dashboard)/guardrails/_components/guardrail_info_helpers.tsx

The request walk redacted a tool call's `arguments` -- plus the legacy `function_call`, Anthropic `tool_use.input` leaves and the Responses API's `function_call` / `function_call_output` fields -- while the response walk restored only `message.content`. A placeholder therefore reached the caller inside a tool call, and nothing raised. This is the same change as the out-of-tree example adapter this file is copied from, kept body-identical on purpose: the response side now collects every restorable span in one positional rehydrate batch, streaming keeps a window per (choice index, tool-call index) and flushes each into the chunk carrying the finish_reason, and `apply_guardrail` restores `inputs["tool_calls"]` on the response side. The declared limit on restoring values inside a JSON string is documented in the module.

greptile-apps · 2026-09-23T01:56:36Z

Confidence Score: 5/5

[High risk] Adds a new guardrail integration that intercepts and modifies request and response payloads.

The PR appears safe to merge based on the latest changes and the current state of the previous threads.

Summary

The PR adds an optional LLM Shield Proxy guardrail for outbound PII redaction and response rehydration.

Covers request fields, non-streaming replies, and chat, Anthropic, and Responses streams.
Adds configuration, dashboard discovery, and tests. The latest changes remove explanatory comments without changing behavior.

_{Reviews (9) · Last reviewed commit: "style(guardrails): drop routine comments..."}

ninadphalak · 2026-10-04T17:11:39Z

@yucheng-berri all the veria and Bugbot findings are addressed:

71e68fd: restored replies and stream chunks are copies, so the response cache keeps placeholders (and the deployment hook no longer restores); Completions choice.text is restored; Responses replay fields (input, code, tool output parts), typed prompt variables and system/developer input items are covered; get_config_model is wired for the dashboard.
974b326: Anthropic plain-text document blocks are redacted.
be16f7b: extra_body overrides are redacted.
ac6af96: test fix (order-dependent import).

CI is green on ac6af96. The three Bugbot threads are fixed in 71e68fd. The only open item is the tool-results design question in its thread, which veria leaves for a maintainer: tool results stay restorable on purpose (reasoning in that thread). Could you make the call on it and take a look when you have a moment?

yucheng-berri · 2026-10-05T07:04:52Z

@greptileai review latest head

yucheng-berri · 2026-10-05T07:04:59Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

… outside the proxy 71e68fd stopped the deployment post-call hook from restoring, so the response cache never holds restored plaintext. Inside the proxy that is right: the proxy's post-call hook restores after the cache write. But with model-level `guardrails` on the SDK, the deployment hooks are the only redact and restore steps, so callers got placeholders back. When the deployment pre-call hook is the one that redacts, it now records the request's vault id and marks the request no-cache / no-store; the deployment post-call hook restores only when that record matches. The cache key there is built from the redacted request and a cache hit skips the post-call hook, so a cached reply could neither be restored nor safely shared. Proxy requests carry no record and keep restoring in the proxy's post-call hook, after the cache write.

yucheng-berri · 2026-10-05T08:19:35Z

@ninadphalak 1 issue I found:

On streams with n≥2 and stream_options.include_usage, the end-of-stream flush in async_post_call_streaming_iterator_hook → _flush_trailing → _chunk_for_choice (llm_shield_proxy.py:1577) deep-copies the last chunk it saw — which by then is the usage-bearing chunk — so the synthetic flush chunk re-emits an identical copy of the usage object alongside the last choice's held-back text. The values are identical so standard OpenAI SDK clients are unaffected, but any consumer that sums usage chunks double-counts the request; a one-line fix is to clear usage on the copy inside _chunk_for_choice.

@yucheng-berri

… flush chunks With n>=2 and stream_options.include_usage, the end-of-stream flush copies the last chunk the stream carried, which is the one holding usage, so each synthetic flush chunk repeated it and a consumer summing usage chunks counted the request twice. The copy now drops `usage`, matching a normal mid-stream chunk. Reported by @yucheng-berri.

yucheng-berri · 2026-10-05T09:00:33Z

@greptileai review latest head

yucheng-berri · 2026-10-05T09:00:38Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

…ry, refuse SDK streams - The post-call restore hook no longer goes through log_guardrail_information, which recorded its whole return value, the restored reply, as guardrail_response. That field is exported to traces even with message logging turned off. - A model-level stream outside the proxy is refused once redacted. Nothing restores an SDK stream, and its cache writer reads the request from before the deployment hook, so it also got cached despite the no-store bypass. - Drop a narrating comment, and keep example_config.yaml to config only; the how-to lives in the docs PR.

…lk and stream modules The module had grown past 1,700 lines. Shared payload types and helpers move to payload.py, the request walk to request_walk.py and the stream restorers to stream_restorers.py; llm_shield_proxy.py keeps the guardrail class. No behaviour change.

AGENTS.md keeps source comments to tool directives and genuinely complex logic; the rationale stays in the docstrings.

yucheng-berri · 2026-10-05T17:12:58Z

@greptileai review latest head

yucheng-berri · 2026-10-05T17:13:04Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit db41912. Configure here.}

yucheng-berri · 2026-10-05T17:33:23Z

lgtm, thx for the contribution

ninadphalak · 2026-10-06T04:09:20Z

Thanks @yucheng-berri for the careful review, the usage-chunk catch, and for syncing the docs page and checking it against a running Shield. If anything comes up with the guardrail once it ships in a release, tag me here and I will pick it up.

…#188) * docs(litellm): point to LiteLLM's built-in llm_shield_proxy guardrail LiteLLM merged the guardrail (BerriAI/litellm#42645) and its docs page (BerriAI/litellm-docs#2070) on 2026-10-05. Our LiteLLM page still said the integration had no dashboard card and no LiteLLM docs page. * docs(litellm): name the merge commit instead of a predicted release

* chore(release): llm-shield-proxy 1.6.11 Patch release so the PyPI page says LiteLLM ships LLM Shield Proxy as a built-in guardrail (BerriAI/litellm#42645) and links LiteLLM's setup page. No code changes since 1.6.10. pii-leak-benchmark stays at 0.5.3. * docs(readme): absolute link to the LiteLLM guide so it works on PyPI

ninadphalak added 24 commits September 3, 2026 11:47

feat(ui): list llm shield in the guardrail garden

8ab969d

Adds the card, preset and logo so operators can pick LLM Shield from the guardrails page the same way as the other partner guardrails.

docs(guardrails): add llm shield example config

2da630d

Shows both modes on one entry. Listing only pre_call redacts the request and then hands the placeholders back to the end user, so the test asserts both hooks are enabled.

feat(ui): use the llm shield brand mark for the guardrail logo

5fccbfe

docs(guardrails): correct the llm shield start command

0de4b8b

fix(guardrails): narrow the stream delta before writing to it

295ad52

basedpyright could not prove the delta was non-None on the write path, and reportOptionalMemberAccess has a zero budget. The guard is also clearer than relying on the text check to imply it.

fix(guardrails): drop Final from a loop-assigned local

f3eb108

basedpyright rejects a Final assigned inside a loop, and reportGeneralTypeIssues sits one over its budget ceiling.

fix(guardrails): narrow prompt and input to a list before iterating

a35c502

Guarding with a conditional iterable left the value un-narrowed, so passing it on was an argument-type error and the element checks read as unreachable. An early return narrows it properly and reads better.

style(guardrails): satisfy ruff format and annotate the new tests

8b2ee5a

`ruff format` wanted the widened `_collect_responses_fields` signature on one line, and the three tests added with the split-vault fix needed return annotations to keep ANN201 level with the base.

ninadphalak requested a review from a team September 23, 2026 01:52

ninadphalak requested review from ryan-crabbe-berri and yuneng-berri as code owners September 23, 2026 01:52

greptile-apps Bot reviewed Sep 23, 2026

View reviewed changes

veria-ai Bot reviewed Sep 23, 2026

View reviewed changes

Comment thread litellm/proxy/guardrails/guardrail_hooks/llm_shield_proxy/llm_shield_proxy.py Outdated

cursor Bot reviewed Oct 5, 2026

View reviewed changes

greptile-apps Bot reviewed Oct 5, 2026

View reviewed changes

Comment thread litellm/proxy/guardrails/guardrail_hooks/llm_shield_proxy/llm_shield_proxy.py Outdated

veria-ai Bot reviewed Oct 5, 2026

View reviewed changes

Comment thread litellm/proxy/guardrails/guardrail_hooks/llm_shield_proxy/llm_shield_proxy.py Outdated

cursor Bot reviewed Oct 5, 2026

View reviewed changes

greptile-apps Bot reviewed Oct 5, 2026

View reviewed changes

ninadphalak added 2 commits October 5, 2026 09:58

greptile-apps Bot reviewed Oct 5, 2026

View reviewed changes

Comment thread litellm/proxy/guardrails/guardrail_hooks/llm_shield_proxy/payload.py Outdated

style(guardrails): drop routine comments from llm_shield_proxy

db41912

AGENTS.md keeps source comments to tool directives and genuinely complex logic; the rationale stays in the docstrings.

cursor Bot reviewed Oct 5, 2026

View reviewed changes

yucheng-berri approved these changes Oct 5, 2026

View reviewed changes

yucheng-berri merged commit d946706 into BerriAI:main Oct 5, 2026
107 checks passed

ninadphalak mentioned this pull request Oct 6, 2026

docs(litellm): point to LiteLLM's built-in llm_shield_proxy guardrail ninadphalak/LLM-Shield-Proxy#188

Merged

ninadphalak mentioned this pull request Oct 6, 2026

chore(release): llm-shield-proxy 1.6.11 ninadphalak/LLM-Shield-Proxy#190

Merged

This was referenced Oct 6, 2026

fix(generic_guardrail_api): return tool_calls on the guardrail response #41013

Open

docs(guardrails): LLM Shield Proxy: which LiteLLM versions include it; quick start runs as written BerriAI/litellm-docs#2138

Open

Conversation

ninadphalak commented Sep 23, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Docs and end-to-end check (Oct 3)

Update — Sep 26

Issue Number

Summary

Changes

Notes for review

Uh oh!

greptile-apps Bot commented Sep 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ninadphalak commented Oct 4, 2026

Uh oh!

yucheng-berri commented Oct 5, 2026

Uh oh!

yucheng-berri commented Oct 5, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

yucheng-berri commented Oct 5, 2026

Uh oh!

yucheng-berri commented Oct 5, 2026

Uh oh!

yucheng-berri commented Oct 5, 2026

Uh oh!

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yucheng-berri commented Oct 5, 2026

Uh oh!

yucheng-berri commented Oct 5, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

yucheng-berri commented Oct 5, 2026

Uh oh!

Uh oh!

ninadphalak commented Oct 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ninadphalak commented Sep 23, 2026 •

edited by cursor Bot

Loading

greptile-apps Bot commented Sep 23, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading