[Bugfix] Opt-in INFO prompt summaries for request logging (--enable-log-request-prompts) by JosephAhn23 · Pull Request #38583 · vllm-project/vllm

JosephAhn23 · 2026-03-30T20:43:29Z

Summary

Addresses #38537 while respecting security posture (see maintainer feedback): truncated prompt previews at INFO are not logged by default. They require an explicit second flag alongside --enable-log-requests.

Behavior

--enable-log-requests alone: INFO logs request id, params, LoRA (same as today); prompt content remains DEBUG only.
--enable-log-requests + --enable-log-request-prompts: INFO logs also include bounded prompt summary (string, token ids, or embed shape), with the same caps as before when --max-log-len is unset.

CLI

New: --enable-log-request-prompts / --no-enable-log-request-prompts (default off).
Validation: prompts flag requires --enable-log-requests.

Code notes

RequestLogger(..., log_prompts_at_info=...)
Gemini review: wording Prompt details; prompt_embeds.shape without redundant tuple(...).

Testing

Please run:

python -m pytest tests/test_logger.py tests/entrypoints/openai/test_cli_args.py -k "log_request_prompt or log_outputs_integration or prompt_at_info" -v

(or full files if preferred).

AI assistance

Developed with AI assistance (Claude). Submitter should review and run tests.

Restore prompt visibility in --enable-log-requests INFO lines when VLLM_LOGGING_LEVEL is INFO (fixes vllm-projectgh-38537). Full payloads remain at DEBUG. Apply default caps when --max-log-len is unset. Co-authored-by: Claude Signed-off-by: Joseph <163171542+JosephAhn23@users.noreply.github.com>

claude

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

github-actions · 2026-03-30T20:43:39Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀

robertgshaw2-redhat · 2026-03-30T20:47:11Z

this is not a regression, it was done on purpose for security reasons (logging client payload is not a good default posture)

Im open to enabling this somehow, but it should not just be via --enable-log-requests

gemini-code-assist

Code Review

This pull request restores visibility for prompt inputs in INFO logs by providing a bounded preview when --max-log-len is unset. It introduces the _prompt_summary_for_info method to handle truncated logging for text, token IDs, and embedding shapes. The review feedback suggests refining the log hint message to more accurately describe the logged prompt details and removing a redundant tuple cast for tensor shapes.

gemini-code-assist · 2026-03-30T20:48:49Z

vllm/entrypoints/logger.py

            logger.info_once(
                "`--enable-log-requests` is set but "
                "the minimum log level is higher than DEBUG. "
-                "Only limited information will be logged to minimize overhead. "
-                "To view more details, set `VLLM_LOGGING_LEVEL=DEBUG`."
+                "Prompt text at INFO is truncated when long; "
+                "set `VLLM_LOGGING_LEVEL=DEBUG` for full details."
            )


The updated hint message specifically mentions 'Prompt text', but the implementation also includes prompt_token_ids and prompt_embeds shape in the INFO logs. To be more accurate and consistent with the changes in _prompt_summary_for_info, the message should refer to 'Prompt details' or 'Prompt inputs'.

Suggested change

logger.info_once(

"`--enable-log-requests` is set but "

"the minimum log level is higher than DEBUG. "

"Only limited information will be logged to minimize overhead. "

"To view more details, set `VLLM_LOGGING_LEVEL=DEBUG`."

"Prompt text at INFO is truncated when long; "

"set `VLLM_LOGGING_LEVEL=DEBUG` for full details."

)

logger.info_once(

"`--enable-log-requests` is set but "

"the minimum log level is higher than DEBUG. "

"Prompt details at INFO are truncated when long; "

"set `VLLM_LOGGING_LEVEL=DEBUG` for full details."

)

gemini-code-assist · 2026-03-30T20:48:49Z

vllm/entrypoints/logger.py

+            preview_ids = prompt_token_ids[:max_ids]
+            return f", prompt_token_ids: {preview_ids}"
+        if prompt_embeds is not None:
+            return f", prompt_embeds: shape={tuple(prompt_embeds.shape)}"


The use of tuple(prompt_embeds.shape) is redundant because prompt_embeds.shape (a torch.Size object) already behaves like a tuple and its string representation is typically sufficient for logging. While not a bug, removing the explicit cast simplifies the code.

Suggested change

return f", prompt_embeds: shape={tuple(prompt_embeds.shape)}"

return f", prompt_embeds: shape={prompt_embeds.shape}"

…ompts Security: do not log client payload snippets at INFO unless explicitly opted in alongside --enable-log-requests (maintainer feedback on vllm-project#38583). - Add FrontendArgs.enable_log_request_prompts / CLI flag - Apply review nits: wording 'Prompt details', use prompt_embeds.shape - Validate flag requires --enable-log-requests - Tests: opt-in prompt logging; CLI validation Co-authored-by: Claude Signed-off-by: Joseph <163171542+JosephAhn23@users.noreply.github.com>

JosephAhn23 · 2026-03-30T20:52:00Z

Thanks @robertgshaw2-redhat — agreed that logging client payloads should not be the default. The branch now gates truncated prompt previews at INFO behind a separate opt-in --enable-log-request-prompts (requires --enable-log-requests). DEBUG behavior is unchanged for full prompt details.

Also applied the gemini-code-assist wording (Prompt details) and dropped the redundant tuple(...) on embed shapes.

Usage: `vllm serve ... --enable-log-requests --enable-log-request-prompts`

Co-authored-by: Claude Signed-off-by: Joseph <163171542+JosephAhn23@users.noreply.github.com>

JosephAhn23 · 2026-03-30T20:59:18Z

Hi @robertgshaw2-redhat, I have updated the PR so prompt logging at INFO is opt-in via `--enable-log-request-prompts` (requires `--enable-log-requests`). I also applied the gemini-code-assist suggestions: Prompt details wording and `prompt_embeds.shape` without the redundant `tuple(...)`. There is also a small ruff line-wrap fix on the validation branch. Ready for whatever you would like next.

JosephAhn23 · 2026-03-30T20:59:45Z

Hi @robertgshaw2, I've updated the PR to make the prompt logging *opt-in* via a new --enable-log-request-prompts flag. I also incorporated the Gemini bot's <#38583> suggestions regarding the 'Prompt details' wording and cleaned up the embedding shape syntax. Ready for the next steps.

…

On Mon, Mar 30, 2026 at 4:47 PM Robert Shaw ***@***.***> wrote: *robertgshaw2-redhat* left a comment (vllm-project/vllm#38583) <#38583?email_source=notifications&email_token=BG44ZVQMYOUFIG2KRDDGTSL4TLMONA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMJVHAYDONJWHE22M4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2LK4DSL5RW63LNMVXHIX3POBSW4X3DNRUWG2Y#issuecomment-4158075695> this is not a regression, it was done on purpose for security reasons (logging client payload is not a good default posture) Im open to enabling this somehow, but it should not just be via --enable-log-requests — Reply to this email directly, view it on GitHub <#38583?email_source=notifications&email_token=BG44ZVQMYOUFIG2KRDDGTSL4TLMONA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMJVHAYDONJWHE22M4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2LK4DSL5RW63LNMVXHIX3POBSW4X3DNRUWG2Y#issuecomment-4158075695>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BG44ZVTLLGUETOKJKLVQG2D4TLMONAVCNFSM6AAAAACXGY5E46VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DCNJYGA3TKNRZGU> . You are receiving this because you authored the thread.Message ID: ***@***.***>

- troubleshooting: request logging flags and link to security guidance - security: new section on logging client payloads - serve_args: commented YAML example for optional flags Co-authored-by: Claude Signed-off-by: Joseph <163171542+JosephAhn23@users.noreply.github.com>

JosephAhn23 · 2026-03-30T21:08:53Z

Docs added (commit on this branch): user-facing notes for the new flag in `docs/usage/troubleshooting.md`, `docs/usage/security.md` (new “Logging and client payloads” section), and commented YAML keys in `docs/configuration/serve_args.md`.

mergify · 2026-03-30T21:09:26Z

Documentation preview: https://vllm--38583.org.readthedocs.build/en/38583/

JosephAhn23 requested a review from njhill as a code owner March 30, 2026 20:43

claude bot reviewed Mar 30, 2026

View reviewed changes

mergify bot added frontend bug Something isn't working labels Mar 30, 2026

gemini-code-assist bot reviewed Mar 30, 2026

View reviewed changes

JosephAhn23 requested review from DarkLight1337, NickLucche, aarnphm, chaunceyjiang, robertgshaw2-redhat and russellb as code owners March 30, 2026 20:52

JosephAhn23 changed the title ~~[Bugfix] Include truncated prompt in INFO request logs when using --enable-log-requests~~ [Bugfix] Opt-in INFO prompt summaries for request logging (--enable-log-request-prompts) Mar 30, 2026

style: wrap long line in validate_parsed_serve_args

5bd70c5

Co-authored-by: Claude Signed-off-by: Joseph <163171542+JosephAhn23@users.noreply.github.com>

JosephAhn23 mentioned this pull request Mar 30, 2026

[Bug]: prompt is missing in Received request only params not prompts #38537

Open

1 task

mergify bot added the documentation Improvements or additions to documentation label Mar 30, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Bugfix] Opt-in INFO prompt summaries for request logging (--enable-log-request-prompts)#38583

[Bugfix] Opt-in INFO prompt summaries for request logging (--enable-log-request-prompts)#38583
JosephAhn23 wants to merge 4 commits intovllm-project:mainfrom
JosephAhn23:fix/38537-log-request-prompt-info

JosephAhn23 commented Mar 30, 2026 •

edited

Loading

Uh oh!

claude bot left a comment

Uh oh!

github-actions bot commented Mar 30, 2026

Uh oh!

robertgshaw2-redhat commented Mar 30, 2026

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot Mar 30, 2026

Uh oh!

gemini-code-assist bot Mar 30, 2026

Uh oh!

JosephAhn23 commented Mar 30, 2026

Uh oh!

JosephAhn23 commented Mar 30, 2026

Uh oh!

JosephAhn23 commented Mar 30, 2026 via email

Uh oh!

JosephAhn23 commented Mar 30, 2026

Uh oh!

mergify bot commented Mar 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	return f", prompt_embeds: shape={tuple(prompt_embeds.shape)}"
	return f", prompt_embeds: shape={prompt_embeds.shape}"

Uh oh!

Conversation

JosephAhn23 commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Behavior

CLI

Code notes

Testing

AI assistance

Uh oh!

claude bot left a comment

Choose a reason for hiding this comment

Claude Code Review

Uh oh!

github-actions bot commented Mar 30, 2026

Uh oh!

robertgshaw2-redhat commented Mar 30, 2026

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

JosephAhn23 commented Mar 30, 2026

Uh oh!

JosephAhn23 commented Mar 30, 2026

Uh oh!

JosephAhn23 commented Mar 30, 2026 via email

Uh oh!

JosephAhn23 commented Mar 30, 2026

Uh oh!

mergify bot commented Mar 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

JosephAhn23 commented Mar 30, 2026 •

edited

Loading