[PD] Rate limit prefill inflight polling warnings

[tangcy98]

Motivation

In disaggregated prefill, process_disagg_prefill_inflight_queue polls inflight KV senders on every scheduler loop. If a request remains in a transient non-terminal polling state, the same warning can be emitted repeatedly for the same rid within a short time.

The request is still treated as undone and can complete normally later, so this warning is useful as a signal but too noisy when printed every loop.

This is the output of the current code in this case.

Modifications

• Use logger.warning_once for prefill inflight unexpected polling state warnings.
• Keep the existing warning semantics, but suppress duplicate identical warning messages.
• Apply the same warning_once behavior to both the PP consensus path and the generic prefill inflight unexpected-state path.

Accuracy Tests

N/A. This PR only changes warning/logging behavior.

Speed Tests and Profiling

N/A. No inference hot-path behavior is changed beyond de-duplicating repeated warning logs.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

[gemini-code-assist]

Code Review

This pull request introduces a mechanism to suppress redundant warning logs during prefill inflight polling by adding a _log_prefill_inflight_poll_warning helper. This helper tracks warning states per request ID on the scheduler to throttle repeated messages. The review feedback suggests improving the readability of the cleanup logic by replacing a getattr call with an explicit hasattr check when removing completed requests from the warning state.

[ShangmingCai]

Why not just use logger.warning_once?

[tangcy98]

I agree that logger.warning_once would be more concise, but I'm unsure whether it is needed here to inform the user about the real-time status of this request. Using warning_once would lose track of the request's status. What do you think? I can update it to a more suitable implementation.

[ShangmingCai]

In fact, in process_disagg_prefill_inflight_queue, the kvpoll state should never beKVPoll.Bootstrapping, I have never seen this warning log before. I assume you are using nixl/mori backend?

[tangcy98]

I am using Dynamo+SGLang+Qwen3.5-122B-A10B+Mooncake in a 4p1d scenario. Here is my start command:

python3 -m dynamo.sglang
--model-path "$PREFILL_MODEL" \
--served-model-name "$MODEL_NAME" \
--trust-remote-code \
--enable-metrics \
--collect-tokens-histogram \
--tp-size 1 \
--quantization w4afp8 \
--kv-cache-dtype fp8_e4m3 \
--mem-fraction-static 0.875 \
--context-length 32768 \
--reasoning-parser qwen3 \
--tool-call-parser qwen3_coder \
--speculative-algorithm NEXTN \
--speculative-num-steps 2 \
--speculative-eagle-topk 1 \
--speculative-num-draft-tokens 3 \
--chunked-prefill-size 16384 \
--max-running-requests 4 \
--max-mamba-cache-size 20 \
--mamba-scheduler-strategy extra_buffer \
--tokenizer-backend fastokens \
--linear-attn-backend flashinfer \
--page-size 64 \
--disable-cuda-graph \
--disaggregation-mode prefill \
--disaggregation-transfer-backend mooncake \
--disaggregation-bootstrap-port "$bootstrap_port" \
"${ib_args[@]}" \
--host 0.0.0.0 \
--port "$sglang_port"

[ShangmingCai]

I think warning_once should be fine. Now we need to figure out why a req could be KVPoll.Bootstrapping status when in process_disagg_prefill_inflight_queue, it shouldn't be at this status anymore when it leaves pop_bootstrapped.

[ShangmingCai]

Please open an issue to explore the bug with the unexpected poll state in process_disagg_prefill_inflight_queue

[tangcy98]

Please open an issue to explore the bug with the unexpected poll state in process_disagg_prefill_inflight_queue

Done. #25063