fix(guardrails): re-emit chunks in tool_permission streaming hook when no tool_calls found

[someswar177]

Relevant issues

Fixes #26547
Re-submission of #26551 (auto-closed when litellm_oss_branch was deleted)

Pre-Submission checklist

• Added test in tests/test_litellm/ — test_async_post_call_streaming_iterator_hook_plain_text_yields_chunks
• make test-unit passes locally; all CI checks pass on this PR
• PR scope isolated to 1 specific problem, 2 files changed
• @greptileai review requested — Confidence Score: 5/5 — "Safe to merge"
• CLA signed

Type

🐛 Bug Fix

Root cause

ToolPermissionGuardrail.async_post_call_streaming_iterator_hook is an async generator (it contains yield statements). In the if not tool_calls: branch — the path taken when the LLM replies with plain text — the original code did a bare return.

In an async generator, return is equivalent to raise StopAsyncIteration. Nothing is yielded. The client receives only data: [DONE] with empty content. The entire plain-text response is silently dropped whenever the tool_permission guardrail is active with mode: post_call on a streaming request.

User-visible impact: Any chat client using LiteLLM proxy with tool_permission guardrail enabled gets a blank reply whenever the LLM decides not to call a tool (a very common case for conversational queries).

Fix

Before returning, re-emit the assembled response through MockResponseIterator — the same pattern already used in the allowed-tool path a few lines below in the same function:

# BEFORE
if not tool_calls:
    verbose_proxy_logger.debug("Tool Permission Guardrail: No tool uses found")
    return

# AFTER
if not tool_calls:
    verbose_proxy_logger.debug("Tool Permission Guardrail: No tool uses found")
    mock_response = MockResponseIterator(model_response=assembled_model_response)
    async for chunk in mock_response:
        yield chunk
    return

Five lines, scoped to the exact location of the bug. Mirrors an existing, well-tested pattern in the same method, so risk of behavioural surprise is minimal.

Screenshots / Proof of Fix

Reproduced against a local LiteLLM proxy (Docker) with the tool_permission guardrail configured in mode: post_call. The same byte-identical streaming request was sent in both runs — the only variable is whether the 5-line fix is present in tool_permission.py.

Reproduction command:

curl -N -X POST http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-1234" \
  -d '{
    "model": "gemini-2.5-flash-lite",
    "stream": true,
    "messages": [{"role":"user","content":"Reply with exactly the word: FOUR. Do not use any tools."}]
  }'

1. Wire-level — Before fix (BUG)

Client receives only data: [DONE]. No content chunk is ever emitted, so the user sees a blank response.

2. Wire-level — After fix (FIX)

Same request, fix applied. A chat.completion.chunk carrying delta.content: "FOUR" is now delivered before [DONE].

3. Server-side control (LiteLLM Logs, from the BUG run)

Even in the bug case, the LLM returned content (ASSISTANT: "FOUR.") and both guardrails passed — the proxy logged the request as Success. This pinpoints the drop: it happens strictly inside async_post_call_streaming_iterator_hook's if not tool_calls: branch, between guardrail evaluation and the SSE response written to the client. That is exactly where this PR's 5 lines live.

Changes

File	Change
litellm/proxy/guardrails/guardrail_hooks/tool_permission.py	+5 lines — the fix
tests/test_litellm/proxy/guardrails/guardrail_hooks/test_tool_permission.py	+40 lines — regression test (test_async_post_call_streaming_iterator_hook_plain_text_yields_chunks) asserts both that ≥1 chunk is yielded and that chunk.delta.content matches the assembled response, guarding against silent drops and content-mangling regressions

PR raised by Someswar at Incubyte

[someswar177]

@greptileai

[greptile-apps]

Greptile Summary

Fixes a silent response-drop bug in ToolPermissionGuardrail.async_post_call_streaming_iterator_hook where a bare return inside an async generator caused the entire plain-text LLM response to be discarded when no tool calls were present, leaving clients with only data: [DONE] and no content.

• tool_permission.py: In the if not tool_calls: branch, adds a MockResponseIterator loop to re-emit the assembled response as streaming chunks before returning — the same five-line pattern already used in the allowed-tool path directly below.
• test_tool_permission.py: Adds a focused regression test that mocks stream_chunk_builder, drives the hook with a plain-text assembled response, and asserts both that ≥1 chunk is yielded and that delta.content matches the expected text, guarding against both dropped-stream and content-corruption regressions.

Confidence Score: 5/5

Safe to merge — a minimal, well-scoped fix that restores plain-text streaming responses in the tool_permission guardrail without touching any other code paths.

The change is exactly five lines in the production file, directly mirrors the already-tested pattern two dozen lines below it in the same method, and is covered by a new regression test that checks both chunk presence and content fidelity. The else fallback for non-ModelResponse types and the None return from stream_chunk_builder are both handled by the existing outer branch structure, so no new edge cases are introduced.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/proxy/guardrails/guardrail_hooks/tool_permission.py	Adds 5 lines to re-emit assembled chunks via MockResponseIterator in the no-tool-calls branch, fixing silent response drops; mirrors the identical pattern already used in the allowed-tools path
tests/test_litellm/proxy/guardrails/guardrail_hooks/test_tool_permission.py	Adds regression test verifying that the hook yields at least one chunk AND that the chunk content matches the assembled response for plain-text (no-tool-call) responses; uses mocks only, no real network calls

_{Reviews (5): Last reviewed commit: "test(guardrails): strengthen plain-text ..." | Re-trigger Greptile}

[greptile-apps]

Loose regression assertion

len(chunks) >= 1 only proves that something was yielded; it doesn't verify that the yielded chunk carries the expected content ("Hello, world!"). A stronger assertion would also confirm that the content attribute in at least one chunk matches the assembled response, making the test a true guard against silent content corruption in addition to the dropped-stream bug.

Suggested change

	assert len(chunks) >= 1, (
	"Hook must yield at least one chunk for plain-text responses; "
	"got none — bare return bug"
	)
	assert len(chunks) >= 1, (
	"Hook must yield at least one chunk for plain-text responses; "
	"got none — bare return bug"
	)
	# Verify the content of the yielded chunks matches the assembled response.
	content = "".join(
	getattr(c.choices[0].delta, "content", "") or ""
	for c in chunks
	if c.choices
	)
	assert "Hello, world!" in content, (
	f"Expected plain-text content to be re-emitted; got: {content!r}"
	)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[someswar177]

@greptileai

[someswar177]

@greptileai

[oss-pr-review-agent-shin]

🤖 litellm-agent: This PR is currently BLOCKED from merge.

Score: 0/5 ❌

Why blocked:

• merge conflicts (rebase against base branch) (merge_conflicts, -5 pts)
• 1 PR-related CI failure (This PR will be auto-closed as it lacks a screenshot for proof of fix. Please include one in the PR description. Add the screenshot-exempt label if this PR has no visible output (e.g. pure docs, CI config).) (pr_related_failures, -2 pts)

Details: Score docked for: merge conflicts (rebase against base branch); 1 PR-related CI failure (This PR will be auto-closed as it lacks a screenshot for proof of fix. Please include one in the PR description. Add the screenshot-exempt label if this PR has no visible output (e.g. pure docs, CI config).).

Fix the issues above and push an update — the bot will re-review automatically.

Note: This bot is still in beta and might not always work as expected. Please share any feedback via Slack.

[someswar177]

@greptileai

[oss-pr-review-agent-shin]

🤖 litellm-agent: This PR is currently BLOCKED from merge.

Score: 3/5 ❌

Why blocked:

• 1 PR-related CI failure (This PR will be auto-closed as it lacks a screenshot for proof of fix. Please include one in the PR description. Add the screenshot-exempt label if this PR has no visible output (e.g. pure docs, CI config).) (pr_related_failures, -2 pts)

Details: Score docked for: 1 PR-related CI failure (This PR will be auto-closed as it lacks a screenshot for proof of fix. Please include one in the PR description. Add the screenshot-exempt label if this PR has no visible output (e.g. pure docs, CI config).).

Fix the issues above and push an update — the bot will re-review automatically.

Note: This bot is still in beta and might not always work as expected. Please share any feedback via Slack.

[someswar177]

@greptileai

[Sameerlite]

LGTM, thanks!