Skip to content

docs(generic_guardrail_api): document fire_and_forget - #2047

Open
caduri wants to merge 2 commits into
BerriAI:mainfrom
caduri:docs/generic-guardrail-fire-and-forget
Open

caduri wants to merge 2 commits into
BerriAI:mainfrom
caduri:docs/generic-guardrail-fire-and-forget

Conversation

@caduri

@caduri caduri commented Oct 3, 2026 •

Copy link
Copy Markdown

Documents the new fire_and_forget and fire_and_forget_max_inflight options for the generic guardrail API, added in BerriAI/litellm#43681. Please merge it only after that PR ships

The page gains both options in the LiteLLM Configuration example and a short "Fire and forget" section with a warning that the guardrail becomes observe-only and can no longer block. The section explains how dispatched calls are recorded (success, so they count as passes in the usage dashboard and compliance checks), that background failures are only logged, that a stream is checked once when it closes, including one the client hangs up on, the per-worker in-flight cap and its not_run drops, the background timeout, and what an invalid value does

npm run build, npm run lint:writing and npm run lint:docs all pass locally

@vercel

vercel Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
litellm Ready Ready Preview Oct 3, 2026 1:48pm UTC

Request Review

@caduri
caduri marked this pull request as ready for review October 4, 2026 11:18

This branch was successfully deployed

1 active deployment
Preview — 3d0b5748 Deployed Oct 3, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant