fix(rollout): abort vLLM rollout via delete-type /abort_requests

[aoshen02]

What

Replace the pause/resume-based rollout abort with a delete-type abort, so that under --partial-rollout the long tail is truncated to partial (and resumable next step) instead of either deadlocking or running to completion.

[read-the-docs-community]

Documentation build overview

📚 vime | 🛠️ Build #33384627 | 📁 Comparing 991473e against latest (429b3d2)

  🔍 Preview build  

41 files changed · + 7 added · ± 34 modified

+ Added

• advanced/delta-weight-sync.html
• advanced/external-rollout-engines.html
• advanced/observability.html
• examples/glm5.2-744B-A40B.html
• _examples_synced/delta_weight_sync/README.html
• _examples_synced/eval_multi_task/README.html
• _examples_synced/mem_agent/README.html

± Modified

• genindex.html
• index.html
• search.html
• advanced/arch-support-beyond-megatron.html
• advanced/fault-tolerance.html
• advanced/low-precision.html
• advanced/megatron-config.html
• advanced/pd-disaggregation.html
• advanced/reproducibility.html
• advanced/speculative-decoding.html
• and 24 more...

[gemini-code-assist]

Code Review

This pull request refactors the abort mechanism for the vLLM rollout backend to drop in-flight requests directly via a new /abort_requests endpoint on the workers, eliminating the need to pause and resume the scheduler. The changes include adding the endpoint to the vLLM patch, implementing the control-plane helper, updating the rollout logic to periodically re-sweep during draining, and adding corresponding unit tests. Feedback on the changes highlights critical robustness issues in the vLLM patch's endpoint, specifically regarding JSON type safety when parsing request bodies, potential string iteration bugs if a single string is passed as request_ids, and import safety when catching JSON decoding errors.

[gemini-code-assist]

There are a few robustness and correctness issues in this section of the patch:

1. JSON Type Safety: raw_request.json() can return any valid JSON type (such as a list, string, number, or boolean). If a client sends a non-dictionary JSON payload, calling body.get(...) will raise an AttributeError: 'list' object has no attribute 'get'. We should check isinstance(body, dict) to prevent this.
2. String Iteration Bug: If request_ids is passed as a single string (e.g., {"request_ids": "req_1"}), iterating over it or passing it directly to engine.abort() might treat it as an iterable of characters (['r', 'e', 'q', '_', '1']), attempting to abort non-existent single-character request IDs. Wrapping a single string in a list prevents this.
3. Import Safety: Catching ValueError instead of json.JSONDecodeError is safer because json.JSONDecodeError inherits from ValueError, and this avoids any potential NameError if json is not imported in api_router.py.

+    try:
+        body = await raw_request.json()
+        if not isinstance(body, dict):
+            body = {}
+    except ValueError:
+        body = {}
+
+    request_ids = body.get("request_ids")
+    if isinstance(request_ids, str):
+        request_ids = [request_ids]
+    elif not request_ids:
+        request_ids = list(engine.output_processor.request_states.keys())

[Josephasafg]

Do you think that in order to keep it less "wasteful" in terms of requests - maybe its worth calling await abort_inflight_requests(urls) once before the loop and then only when asyncio.wait times out?

[aoshen02]

fix

[Josephasafg]

Optional safeguard: the resweep handles transient worker blips and multi-turn stragglers fine, but if an engine crashes/hangs (or the connection goes half-open) the aborted stream never returns and await post - timeout=None - hangs forever, so the drain re-sweeps indefinitely. Could we maybe do something like:

_ABORT_MAX_DRAIN_S = 60

in the main loop

loop = asyncio.get_running_loop()
deadline = loop.time() + _ABORT_MAX_DRAIN_S
while state.pendings and loop.time() < deadline:

and then outside the loop:

if state.pendings:
        for task in state.pendings:
            task.cancel()
        await asyncio.gather(*state.pendings, return_exceptions=True)

[aoshen02]

I think for now we will keep parity with slime for this part. But will open to modify it if bug happens.

[Josephasafg]

LGTM. Thanks! I ran the use-case that failed for me and this fix seems to work.

[CalvinXKY]

LGTM