[talorabr]

Overview

This PR adds an option to change the log level used for logging the request body JSON.
This PR is related to the previously merged PR: #58163

Claims

• Default behavior is not changed: default log level for the request body JSON is DEBUG when --enable-log-requests is set.
• Setting VLLM_REQUEST_BODY_LOG_LEVEL=INFO with --enable-log-requests makes all requests' JSON body will be logged in INFO log level.
• Setting VLLM_REQUEST_BODY_LOG_LEVEL=NONEXIST will show a warning to the user that this log level does not exist, and request body logging will be skipped.

Validation

In all validations I sent the following request to the server:

curl --silent --show-error --request POST http://$VLLM_SERVER_URL:8000/v1/chat/completions -H 'Content-Type: application/json' --data   '{"model": "meta-llama/Llama-3.2-1B-Instruct", "messages": [ { "role": "system", "content": "You are a concise assistant." }, { "role": "user", "content": "Say hello in exactly five words." } ], "temperature": 0, "max_tokens": 16, "stream": false, "top_p": 0.95  }'

Default behavior is not changed

Server command - vllm serve --model meta-llama/Llama-3.2-1B-Instruct --host 0.0.0.0 --trust-remote-code --dtype auto --gpu-memory-utilization 0.40 --enforce-eager

Log -

INFO:     Started server process [2966199]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
(APIServer pid=2966199) WARNING 10-08 08:18:35 [vllm.py:1327] Watermarking is enabled for this request, but the engine has no watermark configuration. This and subsequent requests will run without watermarking.
INFO

Server command - vllm serve --model meta-llama/Llama-3.2-1B-Instruct --host 0.0.0.0 --trust-remote-code --dtype auto --gpu-memory-utilization 0.40 --enforce-eager --enable-log-requests

Log -

INFO:     Started server process [2964257]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
(APIServer pid=2964257) INFO 10-08 08:16:48 [request_logger.py:66] Received request chatcmpl-91eb92071857b3f9: params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.0, watermarking=None, top_p=1.0, top_k=0, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[], thinking_token_budget=None, include_stop_str_in_output=False, ignore_eos=False, max_tokens=16, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, structured_outputs=None, extra_args=None), lora_request: None.
(APIServer pid=2964257) WARNING 10-08 08:16:48 [vllm.py:1327] Watermarking is enabled for this request, but the engine has no watermark configuration. This and subsequent requests will run without watermarking.
(APIServer pid=2964257) INFO 10-08 08:16:48 [async_llm.py:568] Added request chatcmpl-91eb92071857b3f9-8a05c170.
INFO:     10.65.35.87:60171 - "POST /v1/chat/completions HTTP/1.1" 200 OK

Server command - VLLM_LOGGING_LEVEL=DEBUG vllm serve --model meta-llama/Llama-3.2-1B-Instruct --host 0.0.0.0 --trust-remote-code --dtype auto --gpu-memory-utilization 0.40 --enforce-eager --enable-log-requests

Log -

INFO:     Started server process [2968123]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
(APIServer pid=2968123) DEBUG 10-08 08:20:29 [v1/metrics/loggers.py:329] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cacheusage: 0.0%
(APIServer pid=2968123) DEBUG 10-08 08:20:30 [entrypoints/.../utils/request_logger.py:83] Request 939a23293148fe01 JSON body: {"messages":[{"content":"You are a concise assistant.","role":"system"},{"content":"Say hello in exactly five words.","role":"user"}],"model":"meta-llama/Llama-3.2-1B-Instruct","max_tokens":16,"stream":false,"temperature":0.0,"top_p":0.95}
(APIServer pid=2968123) DEBUG 10-08 08:20:30 [entrypoints/.../utils/request_logger.py:56] Request chatcmpl-939a23293148fe01 details: prompt: '<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nCutting Knowledge Date: December 2023\nToday Date: 08 Oct 2026\n\nYou are a concise assistant.<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nSay hello in exactly five words.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n', prompt_token_ids: [128000, 128006, 9125, 128007, 271, 38766, 1303, 33025, 2696, 25, 6790, 220, 2366, 18, 198, 15724, 2696, 25, 220, 2318, 5020, 220, 2366, 21, 271, 2675, 527, 264, 64694, 18328, 13, 128009, 128006, 882, 128007, 271, 46864, 24748, 304, 7041, 4330, 4339, 13, 128009, 128006, 78191, 128007, 271], prompt_embeds shape: None.
(APIServer pid=2968123) INFO 10-08 08:20:30 [entrypoints/.../utils/request_logger.py:66] Received request chatcmpl-939a23293148fe01: params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.0, watermarking=None, top_p=1.0, top_k=0, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[],thinking_token_budget=None, include_stop_str_in_output=False, ignore_eos=False, max_tokens=16, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, structured_outputs=None, extra_args=None), lora_request: None.
(APIServer pid=2968123) WARNING 10-08 08:20:30 [config/vllm.py:1327] Watermarking is enabled for this request, but the engine has no watermark configuration. This and subsequent requests will run without watermarking.
(APIServer pid=2968123) INFO 10-08 08:20:30 [v1/engine/async_llm.py:568] Added request chatcmpl-939a23293148fe01-b805cbcb.
(EngineCore pid=2968366) DEBUG 10-08 08:20:30 [v1/engine/core.py:1533] EngineCore loop active.
(EngineCore pid=2968366) DEBUG 10-08 08:20:30 [v1/engine/core.py:1521] EngineCore waiting for work.
INFO:     10.65.35.87:60784 - "POST /v1/chat/completions HTTP/1.1" 200 OK

Setting VLLM_REQUEST_BODY_LOG_LEVEL=INFO with --enable-log-requests

Server command - VLLM_REQUEST_BODY_LOG_LEVEL=INFO vllm serve --model meta-llama/Llama-3.2-1B-Instruct --host 0.0.0.0 --trust-remote-code --dtype auto --gpu-memory-utilization 0.40 --enforce-eager --enable-log-requests

Log -

INFO:     Started server process [2969917]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
(APIServer pid=2969917) INFO 10-08 08:22:20 [request_logger.py:83] Request 8c4f475c52e97174 JSON body: {"messages":[{"content":"You are a concise assistant.","role":"system"},{"content":"Say hello in exactly five words.","role":"user"}],"model":"meta-llama/Llama-3.2-1B-Instruct","max_tokens":16,"stream":false,"temperature":0.0,"top_p":0.95}
(APIServer pid=2969917) INFO 10-08 08:22:20 [request_logger.py:66] Received request chatcmpl-8c4f475c52e97174: params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.0, watermarking=None, top_p=1.0, top_k=0, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[], thinking_token_budget=None, include_stop_str_in_output=False, ignore_eos=False, max_tokens=16, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, structured_outputs=None, extra_args=None), lora_request: None.
(APIServer pid=2969917) WARNING 10-08 08:22:20 [vllm.py:1327] Watermarking is enabled for this request, but the engine has no watermark configuration. This and subsequent requests will run without watermarking.
(APIServer pid=2969917) INFO 10-08 08:22:20 [async_llm.py:568] Added request chatcmpl-8c4f475c52e97174-ac72befd.
INFO:     10.65.35.87:64285 - "POST /v1/chat/completions HTTP/1.1" 200 OK

Setting VLLM_REQUEST_BODY_LOG_LEVEL=NONEXIST will show a warning

Server command - VLLM_REQUEST_BODY_LOG_LEVEL=NONEXIST vllm serve --model meta-llama/Llama-3.2-1B-Instruct --host 0.0.0.0 --trust-remote-code --dtype auto --gpu-memory-utilization 0.40 --enforce-eager --enable-log-requests

Log -

INFO:     Started server process [2972353]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
(APIServer pid=2972353) WARNING 10-08 08:24:55 [request_logger.py:77]Invalid logging level for request content: NOTEXIST. Request JSON body will not be logged.
(APIServer pid=2972353) INFO 10-08 08:24:55 [request_logger.py:66] Received request chatcmpl-9dd59f45aa8e4cc5: params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.0, watermarking=None, top_p=1.0, top_k=0, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[], thinking_token_budget=None, include_stop_str_in_output=False, ignore_eos=False, max_tokens=16, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, structured_outputs=None, extra_args=None), lora_request: None.
(APIServer pid=2972353) WARNING 10-08 08:24:55 [vllm.py:1327] Watermarking is enabled for this request, but the engine has no watermark configuration. This and subsequent requests will run without watermarking.
(APIServer pid=2972353) INFO 10-08 08:24:55 [async_llm.py:568] Added request chatcmpl-9dd59f45aa8e4cc5-8a603b79.
INFO:     10.65.35.87:62996 - "POST /v1/chat/completions HTTP/1.1" 200 OK

Details

Motivation: Previously, logging just the request JSON body would require turning on the DEBUG log, which can be very heavy. In some cases, like running agentic benchmarks, we want to gather the requests for an option to replay these requests.

Design decisions:
The user can choose the log level for the request JSON body logging using an environment variable, while maintaining backward compatibility with the DEBUG log that was the behavior before.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[benchislett]

/ci run

[github-actions]

✅ @talorabr, CI is now available for this PR.

• /ci run starts upstream CI; /amd-ci run starts AMD CI only.
• Run commands test your branch with every commit currently on its upstream target branch. If it is behind, the bot merges the target branch into it first (pull before pushing again). Append --allow-stale to a run command to test an outdated branch as is, at your own risk.
• /ci retry retries failed jobs in the CI build for the current PR head. If the current head has no CI build, it starts a new CI build for the current head containing only jobs that failed in the latest earlier CI build for this PR.
• /amd-ci retry retries failed jobs in AMD CI for the current PR head. Use /amd-ci run when the current head has no AMD CI build.
• /ci cancel cancels scheduled or running CI builds for this PR branch; /amd-ci cancel does the same for AMD CI only.

[github-actions]

✅ Triggered Buildkite CI #94945 for commit 22006702086e.

[talorabr]

/ci retry

[github-actions]

✅ Queued 1 failed job(s) for retry in Buildkite CI #94945.

[talorabr]

/ci retry

[github-actions]

✅ Buildkite CI #94945 failed during CI setup, so its test failure set is incomplete. Use /ci run for the new commit.

[talorabr]

/ci run

[github-actions]

✅ Triggered Buildkite CI #95097 for commit 266a2c56fc1f.

ℹ️ This PR is 3 commits behind upstream main. Merged upstream main into this branch as 266a2c56fc1f before starting CI. Pull the branch before pushing more changes.

[markmc]

This should have been added to the recently added --logging-config, see #57200

We're trying to limit the proliferation of env vars, see #25700