[Feature] Routing replay (R3) for vLLM rollout (vLLM 0.22+, /inference/v1/generate) by CalvinXKY · Pull Request #49 · vllm-project/vime

CalvinXKY · 2026-05-27T07:28:10Z

Purpose

Implements RFC #32 Phase 2 / follow-up to #34: MoE routing replay (--use-rollout-routing-replay) on the vLLM rollout path using vLLM ≥ 0.22 native support (vllm#39568).

SGLang already fills sample.rollout_routed_experts with shape (len(tokens) - 1, num_layers, moe_router_topk). On vLLM 0.21.x, routing required a site-packages patch and/or /v1/completions workarounds. From vLLM 0.22+, /inference/v1/generate returns routed experts (often as a single base64 npy buffer on choices[].routed_experts).

This PR drops the disagg patch path and standardizes rollout + engine smoke checks on /inference/v1/generate.

Supersedes the approach in #34 (v0.21.0 + docker/patch/latest/vllm.patch + completions fallback).

What's included

Rollout (`slime/rollout/vllm_rollout.py`)

R3 text generation uses POST /inference/v1/generate only (removed /v1/completions branch).
_merge_generate_routed_experts: merge split or single-buffer routing; align to len(tokens) - 1.

Engine (`slime/backends/vllm_utils/vllm_engine.py`)

When use_rollout_routing_replay: inject --enable-return-routed-experts only (no 0.21.x --no-async-scheduling / --no-enable-prefix-caching guards).
_verify_generate_routed_experts: startup smoke on /inference/v1/generate.

Tests

tests/unit/rollout/test_vllm_rollout.py — merge/apply routing, single-buffer case.
tests/unit/backends/vllm_utils/test_vllm_engine.py — _verify_generate_routed_experts.
tests/test_vllm_generate_endpoint.py — qwen3-30b-a3b-r3 integration.

Requirements

Item	Notes
vLLM	≥ 0.22.0 (or dev build with #39568)
MoE multi-GPU	`--vllm-enable-expert-parallel`
R3	`--use-rollout-routing-replay`

Verification logs

Environment: 8×A100, container vime_v22, Qwen3-30B-A3B, train/rollout 4+4, vLLM 0.21.1rc1.dev38 (includes #39568 generate routing). No Split sizes / missing-routing errors observed.

1) Local unit tests

pytest tests/unit/rollout/test_vllm_rollout.py tests/unit/backends/vllm_utils/test_vllm_engine.py -q
# 87 passed in 3.08s

pre-commit run --all-files
# all hooks Passed (ruff, black, isort, …)

2) Full verify — HTTP + rollout E2E

Phase A — standalone vLLM HTTP smoke (:8000)

[routing-replay-full] Starting standalone vLLM on http://127.0.0.1:8000 (GPUs=0,1,2,3, tp=4) ...
[routing-replay-full] HTTP model id: /data/nfs_87/model/Qwen3-30B-A3B
[vllm-generate-routing] POST http://127.0.0.1:8000/inference/v1/generate
  routing rows=12 layers=48 top_k=8
OK: /inference/v1/generate routing replay fields present.
[routing-replay-full] HTTP phase PASS

Phase B — debug-rollout-only + .pt shape check

(VLLMEngine) non-default args: {..., 'enable_return_routed_experts': True, 'enable_expert_parallel': True, ...}
(EngineCore) RoutedExpertsManager CPU buffer: 0.63 GB (slots=1642320, layers=48, top_k=8, dtype=uint8)
(APIServer) Route: /inference/v1/generate, Methods: POST
(APIServer) "POST /inference/v1/generate HTTP/1.1" 200 OK

  sample[0]: OK shape=(322, 48, 8) tokens=323 response_length=128
  sample[1]: OK shape=(265, 48, 8) tokens=266 response_length=128
PASS: 2 sample(s) satisfy routing replay shape contract.
[routing-replay-verify] Rollout path PASS

3) Production-shaped training — 4+4 R3 smoke

(VLLMEngine) non-default args: {..., 'enable_return_routed_experts': True,
  'tensor_parallel_size': 4, 'enable_expert_parallel': True, ...}
(Worker) Initializing routed experts capturer, enable_return_routed_experts: True
(APIServer) Route: /inference/v1/generate, Methods: POST
(APIServer) "POST /inference/v1/generate HTTP/1.1" 200 OK

(MegatronTrainRayActor) torch.from_numpy(r) for r in rollout_data["rollout_routed_experts"]
(RolloutManager) perf 0: {..., 'perf/tokens_per_gpu_per_sec': 305.09, ...}
(MegatronTrainRayActor) perf 0: {..., 'perf/actor_train_tok_per_s': 687.56, 'perf/step_time': 223.28, ...}

Training completed multiple rollout→train steps without routing-shape failures.

Test plan

Unit

pytest tests/unit/rollout/test_vllm_rollout.py -k "routed or merge_generate"
pytest tests/unit/backends/vllm_utils/test_vllm_engine.py -k verify_generate

Integration (GPU)

pytest tests/test_vllm_generate_endpoint.py -k r3

Manual

vLLM ≥ 0.22 + --enable-return-routed-experts (no patch).
--use-rollout-routing-replay --vllm-enable-expert-parallel, 4+4 layout.
Confirm rollout_routed_experts shape (len(tokens)-1, num_layers, moe_router_topk).

Out of scope

Docker base bump to v0.22.0 image tag.
Colocate CI parity with tests/test_qwen3_30B_A3B_r3.py.
Removing legacy docker/patch tree.

Code Review

This pull request updates the vLLM backend and rollout components to support MoE routing replay with vLLM 0.22+. It introduces logic to merge prompt and generation routed experts from the /inference/v1/generate endpoint, align routing rows, and perform smoke checks during engine initialization. It also consolidates weight transfer HTTP timeout retrieval and updates unit tests. The reviewer identified several opportunities to clean up the codebase, including removing unused helper functions (_response_json_or_fallback and _routing_rows_from_http_payload) and their associated imports, as well as eliminating a duplicate helper function (_encode_routed_npy) in the test suite.

gemini-code-assist · 2026-05-27T07:29:50Z

+from typing import TYPE_CHECKING, Any
+
+if TYPE_CHECKING:
+    import numpy as np


Since _routing_rows_from_http_payload is unused and can be removed, the TYPE_CHECKING block and the Any import are no longer needed. Removing them keeps the imports clean.

make sense, could you remove this?

gemini-code-assist · 2026-05-27T07:29:50Z

+def _response_json_or_fallback(response) -> dict:
+    """Parse JSON from an HTTP response, returning a structured error dict on failure."""
+    try:
+        data = response.json()
+    except (ValueError, json.JSONDecodeError):
+        return {"ok": False, "error": "Invalid JSON response", "raw": getattr(response, "text", "")}
+    if not isinstance(data, dict):
+        return {"ok": False, "error": "Response is not a dictionary", "data": data}
+    return data
+
+
+def _routing_rows_from_http_payload(value: Any) -> np.ndarray | None:
+    """Decode vLLM routed-experts HTTP field (base64 npy or nested list)."""
+    import base64
+    import io
+
+    import numpy as np
+
+    if value is None:
+        return None
+    if isinstance(value, str):
+        return np.load(io.BytesIO(base64.b64decode(value)), allow_pickle=False)
+    if isinstance(value, list):
+        return np.asarray(value, dtype=np.int32)
+    return None


The helper functions _response_json_or_fallback and _routing_rows_from_http_payload are defined but never called anywhere in the codebase. Removing this dead code improves maintainability and readability.

gemini-code-assist · 2026-05-27T07:29:50Z

+def _encode_routed_npy(arr: np.ndarray) -> str:
    buf = io.BytesIO()
    np.save(buf, arr)
-    encoded = base64.b64encode(buf.getvalue()).decode("ascii")
+    return base64.b64encode(buf.getvalue()).decode("ascii")


The helper function _encode_routed_npy is identical to _encode_routed defined at line 129 of the same file. To avoid code duplication, please remove _encode_routed_npy and reuse _encode_routed instead.

CalvinXKY · 2026-05-27T07:30:04Z

E2E test：

qwen3-30B-A3B-4t4i-r3-300step.sh

aoshen02 · 2026-05-27T08:01:10Z

+                    "Ensure vLLM 0.22+ serves with --enable-return-routed-experts."
+                )
+            train_data["rollout_routed_experts"] = routed
+        elif samples[0].rollout_routed_experts is not None:


Kinda weird, can we avoid passing rollout_routed_experts when use_rollout_routing_replay is false? I think it's inconsistent.

Kinda weird, can we avoid passing rollout_routed_experts when use_rollout_routing_replay is false? I think it's inconsistent.

Good catch, removed

aoshen02 · 2026-05-27T08:02:04Z

        _, action = entry
        return getattr(args, dest, action.default) != action.default

+    # MoE routing replay (vLLM 0.22+, PR #39568): routed experts on ``/inference/v1/generate``.


I don't think we need to move the code

I don't think we need to move the code

Agreed

aoshen02 · 2026-05-27T08:02:52Z

+    try:
+        data = response.json()
+    except (ValueError, json.JSONDecodeError):
+        return {"ok": False, "error": "Invalid JSON response", "raw": getattr(response, "text", "")}


aoshen02 · 2026-05-27T08:04:48Z

+    return None
+
+
+def _verify_generate_routed_experts(base_url: str, model: str, timeout_s: float = 120.0) -> None:


I’m not sure _verify_generate_routed_experts() belongs in VLLMEngine.init(). I think we should move it to integration test

I’m not sure _verify_generate_routed_experts() belongs in VLLMEngine.init(). I think we should move it to integration test

That's a fair point — the main reason I kept it in init() is to fail fast at engine startup rather than discovering a broken routing configuration only when the first rollout batch arrives.

I think this belongs in a smoke/integration test rather than production init code. rollout already validates missing rollout_routed_experts before training, so we still fail on the first real rollout anyway.

aoshen02 · 2026-05-27T08:22:33Z

+        args,
+        sample,
+        output,
+        choice,


One issue with the current ordering: _apply_vllm_routed_experts() checks sample.status == Sample.Status.ABORTED to allow abort + 0-token samples without routed experts, but sample.update_from_meta_info(args, meta) is called only after _apply_vllm_routed_experts().

So if vLLM returns finish_reason="abort" with no generated tokens/routed_experts, the sample is still PENDING when _apply_vllm_routed_experts() runs, and the intended abort guard will not trigger. This can raise RuntimeError during partial rollout abort/drain.

Could we either call sample.update_from_meta_info(args, meta) before _apply_vllm_routed_experts(), or make _apply_vllm_routed_experts() check the parsed finish reason directly instead of relying on sample.status?

One issue with the current ordering: _apply_vllm_routed_experts() checks sample.status == Sample.Status.ABORTED to allow abort + 0-token samples without routed experts, but sample.update_from_meta_info(args, meta) is called only after _apply_vllm_routed_experts().

So if vLLM returns finish_reason="abort" with no generated tokens/routed_experts, the sample is still PENDING when _apply_vllm_routed_experts() runs, and the intended abort guard will not trigger. This can raise RuntimeError during partial rollout abort/drain.

Could we either call sample.update_from_meta_info(args, meta) before _apply_vllm_routed_experts(), or make _apply_vllm_routed_experts() check the parsed finish reason directly instead of relying on sample.status?

Fixed by calling sample.update_from_meta_info() before _apply_vllm_routed_experts(), so the abort guard based on sample.status works correctly. Also added a regression test.

aoshen02 · 2026-05-27T08:23:15Z

+from typing import TYPE_CHECKING, Any
+
+if TYPE_CHECKING:
+    import numpy as np


make sense, could you remove this?

aoshen02 · 2026-05-27T08:31:21Z

+    """Decode vLLM routed-experts HTTP field (base64 npy or nested list)."""
+    import base64
+    import io
+


Dead code

Done

aoshen02 · 2026-05-27T08:46:40Z

-    vLLM ``/inference/v1/generate`` returns routed experts as a base64 encoded
-    ``.npy`` payload on each response choice when the server is launched with
-    ``--enable-return-routed-experts``.
+def _merge_generate_routed_experts(


Looking at vLLM’s /inference/v1/generate implementation, the response contract for routed experts seems much simpler than the compatibility logic here.

The protocol defines choices[].routed_experts as a single str | None, documented as base64-encoded .npy bytes with decoded shape (num_tokens - 1, num_layers, num_experts_per_tok):
https://github.com/vllm-project/vllm/blob/71d810bbf44b34f3a019730a6878fbcbf2480499/vllm/entrypoints/serve/disagg/protocol.py#L157-L172

The serving code also always encodes it with np.save(...) + base64, and puts it only on choice.routed_experts:
https://github.com/vllm-project/vllm/blob/71d810bbf44b34f3a019730a6878fbcbf2480499/vllm/entrypoints/serve/disagg/serving.py#L277-L294

The vLLM test decodes it the same way:
https://github.com/vllm-project/vllm/blob/71d810bbf44b34f3a019730a6878fbcbf2480499/tests/entrypoints/serve/disagg/test_return_routed_experts.py#L70-L75

Given that contract, do we need to support nested-list payloads, prompt_routed_experts, or split/trim/concat logic in VIME? I think this path can be simplified to only read choice["routed_experts"], decode the base64 .npy, validate ndim == 3, and require the row count to match len(sample.tokens) - 1.

Looking at vLLM’s /inference/v1/generate implementation, the response contract for routed experts seems much simpler than the compatibility logic here.

The protocol defines choices[].routed_experts as a single str | None, documented as base64-encoded .npy bytes with decoded shape (num_tokens - 1, num_layers, num_experts_per_tok): https://github.com/vllm-project/vllm/blob/71d810bbf44b34f3a019730a6878fbcbf2480499/vllm/entrypoints/serve/disagg/protocol.py#L157-L172

The serving code also always encodes it with np.save(...) + base64, and puts it only on choice.routed_experts: https://github.com/vllm-project/vllm/blob/71d810bbf44b34f3a019730a6878fbcbf2480499/vllm/entrypoints/serve/disagg/serving.py#L277-L294

The vLLM test decodes it the same way: https://github.com/vllm-project/vllm/blob/71d810bbf44b34f3a019730a6878fbcbf2480499/tests/entrypoints/serve/disagg/test_return_routed_experts.py#L70-L75

Given that contract, do we need to support nested-list payloads, prompt_routed_experts, or split/trim/concat logic in VIME? I think this path can be simplified to only read choice["routed_experts"], decode the base64 .npy, validate ndim == 3, and require the row count to match len(sample.tokens) - 1.

Simplified to match vLLM's documented contract

aoshen02 · 2026-05-27T08:51:41Z

+        return
+    if sample.status == Sample.Status.ABORTED and sample.response_length == 0:
+        return
+    pre = output.get("prompt_routed_experts")


vllm now doesn't have prompt_router experts so I think we can just simplify this logic.

vllm now doesn't have prompt_router experts so I think we can just simplify this logic.

Simplified to match vLLM's documented contract

aoshen02 · 2026-05-27T08:51:52Z

        gen_url = f"{base}/inference/v1/generate"
        with trace_span(sample, "vllm_mm_generate", attrs={"max_tokens": params["max_new_tokens"]}):
            output = await post(gen_url, generate_body, headers=headers)
+        request_prompt_len = len(generate_body.get("token_ids") or [])


vllm now doesn't have prompt_router experts so I think we can just simplify this logic.

vllm now doesn't have prompt_router experts so I think we can just simplify this logic.

Simplified to match vLLM's documented contract

Address non-controversial PR feedback by deleting unused vLLM engine helper code and reusing the existing routed-experts encoder helper in rollout unit tests.

aoshen02 · 2026-05-27T12:37:35Z

+    sample.rollout_routed_experts = np.ascontiguousarray(arr.astype(np.int32, copy=True))
+
+
+def _vllm_expected_routed_rows_from_tokens(token_count: int) -> int:


Done. Also cleaned up some dead code left over from debugging.

aoshen02 · 2026-05-27T12:44:12Z

+@pytest.mark.unit
+def test_verify_generate_routed_experts_accepts_single_buffer(monkeypatch):
+    prompt_toks = 5
+    gen_toks = 3


We can remove the unit test and have an integration test about it.

After: 254056f

E2E test works well：

aoshen02 · 2026-05-27T15:22:37Z

        )
-        _wait_server_healthy(self._http_base(), process=self.process)
+        base = self._http_base()
+        _wait_server_healthy(base, process=self.process)


I think we can revert here.

CalvinXKY · 2026-05-28T07:07:10Z

CMP Test：

…e/v1/generate) (#49) * vllm engine supports router replay * feat(r3): vLLM 0.22+ generate API for MoE routing replay * chore(review): remove dead helpers and duplicate test encoder Address non-controversial PR feedback by deleting unused vLLM engine helper code and reusing the existing routed-experts encoder helper in rollout unit tests. * remove dead code * refactor(r3): trim redundant routed replay code --------- Co-authored-by: aoshen02 <aoshen@inferact.ai>

…ounterpart) This test was vime-specific (created in #49, not from slime) and was never registered in CI. Remove it to keep the test tree aligned with slime@44d29ee. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…ms (#218) * fix: complete slime #1985 sync — remove TIGHT_HOST_MEMORY + align params PR #214 synced #1985's TIGHT_DEVICE_MEMORY removal and batch-size shrinks but missed 10 files that used TIGHT_HOST_MEMORY (a different env-var guard with the same pre-#1985 pattern). Also aligns n-samples-per-prompt (8→4) and num-critic-only-steps (3→2) that were left at pre-#1985 values. Files: test_moonlight_16B_A3B{,_r3}, test_qwen3_{0.6B_parallel_check, 30B_A3B{,_r3}, 4B_ckpt, 4B_ppo{,_disaggregate,_train_critic_only}, 4B_streaming_partial_rollout}, test_qwen2.5_0.5B_ppo_critic_only_short Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove leftover blank lines from TIGHT_ variable removal #214 and the preceding commit removed TIGHT_DEVICE_MEMORY and TIGHT_HOST_MEMORY definitions but left behind an extra blank line in the header area of 11 test files. Collapse double-blank to single-blank to match slime's spacing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * revert: remove pre-ported slime #2016 content from http_utils http_utils.py had get_rollout_num_engines() and run_router improvements that were pre-ported from slime #2016 (post-cutoff 44d29ee). Revert to match slime@44d29ee baseline so the diff stays clean; #2016 will be synced as a whole when its turn comes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: remove test_vllm_generate_endpoint.py (vime-native, no slime counterpart) This test was vime-specific (created in #49, not from slime) and was never registered in CI. Remove it to keep the test tree aligned with slime@44d29ee. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * sync: port gpu_lock_exec subprocess signal forwarding (slime #1945) PR #155 synced #1945's DistOptim checkpoint rider but missed the gpu_lock_exec.py change: replaces os.execvp with subprocess.Popen + proper signal forwarding (SIGINT/SIGTERM/SIGHUP) and fd_lock cleanup. This prevents orphaned GPU-holding processes when CI runners are cancelled. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: complete #1985 sync + fix NUM_GPUS placement + pre-commit - Remove TIGHT_HOST_MEMORY from 10 test files (missed in #214) - Align n-samples-per-prompt (8→4) and num-critic-only-steps (3→2) - Remove leftover blank lines from TIGHT_ variable removal (11 files) - Revert http_utils.py pre-ported #2016 content to slime@44d29ee baseline - Remove test_vllm_generate_endpoint.py (vime-native, no slime counterpart) - Port gpu_lock_exec subprocess signal forwarding (slime #1945) - Align remaining test params (over-sampling-batch-size, max-tokens-per-gpu) - Full test_qwen3_4B_ckpt.py #1945 sync (optimizer placement CLI) - Fix NUM_GPUS=0 placement in 7 CPU test files (was inside decorators) - NamedTemporaryFile multi-line formatting All pre-commit checks pass. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

CalvinXKY added 2 commits May 27, 2026 15:10

vllm engine supports router replay

7350db3

feat(r3): vLLM 0.22+ generate API for MoE routing replay

d7b22f1

CalvinXKY requested review from andakai and aoshen02 and removed request for aoshen02 May 27, 2026 07:28

gemini-code-assist Bot reviewed May 27, 2026

View reviewed changes

CalvinXKY requested a review from aoshen02 May 27, 2026 07:30

aoshen02 reviewed May 27, 2026

View reviewed changes

CalvinXKY force-pushed the feature/r3_latest branch from ff5b248 to af58b9d Compare May 27, 2026 09:06

chore(review): remove dead helpers and duplicate test encoder

ec0814c

Address non-controversial PR feedback by deleting unused vLLM engine helper code and reusing the existing routed-experts encoder helper in rollout unit tests.

CalvinXKY force-pushed the feature/r3_latest branch from af58b9d to ec0814c Compare May 27, 2026 09:16

aoshen02 reviewed May 27, 2026

View reviewed changes

remove dead code

254056f

aoshen02 reviewed May 27, 2026

View reviewed changes

refactor(r3): trim redundant routed replay code

d9675db

aoshen02 mentioned this pull request May 28, 2026

[Bug] vLLM IPC weight transfer sender still on 2-tuple protocol after vllm#37476; receiver expects bare 15-tuple #58

Closed

aoshen02 merged commit 971520c into main May 28, 2026
10 of 13 checks passed

aoshen02 deleted the feature/r3_latest branch May 28, 2026 07:20

CalvinXKY mentioned this pull request May 28, 2026

[Spike] Phased Qwen3-30B-A3B #14

Closed

This was referenced May 27, 2026

tests + CI: complete sglang→vllm rename across tests/ and .github/ #40

Merged

fix(vllm): unify weight-transfer HTTP handling #47

Merged

CalvinXKY mentioned this pull request Jul 30, 2026

[Ascend][RFC] Mainline Capability Adaptation Checklist (Phase 1 + Phase 2) #279

Open

87 tasks

		return None


		def _verify_generate_routed_experts(base_url: str, model: str, timeout_s: float = 120.0) -> None:

		sample.rollout_routed_experts = np.ascontiguousarray(arr.astype(np.int32, copy=True))


		def _vllm_expected_routed_rows_from_tokens(token_count: int) -> int:

Uh oh!

Conversation

CalvinXKY commented May 27, 2026

Purpose

What's included

Rollout (slime/rollout/vllm_rollout.py)

Engine (slime/backends/vllm_utils/vllm_engine.py)

Tests

Requirements

Verification logs

1) Local unit tests

2) Full verify — HTTP + rollout E2E

3) Production-shaped training — 4+4 R3 smoke

Test plan

Unit

Integration (GPU)

Manual

Out of scope

Related

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot May 27, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 27, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 27, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

CalvinXKY commented May 27, 2026

E2E test：

Uh oh!

aoshen02 May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aoshen02 May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Rollout (`slime/rollout/vllm_rollout.py`)

Engine (`slime/backends/vllm_utils/vllm_engine.py`)

aoshen02 May 27, 2026 •

edited

Loading

aoshen02 May 27, 2026 •

edited

Loading