scripts: complete slime-exact port of most scripts except for gpt-oss 20B support - #260
Merged
CalvinXKY merged 12 commits intoJun 25, 2026
Merged
Conversation
Restore files that were either deleted by vllm-project#126 ("trim examples to qwen3 only") or never synced from slime: **Reverted from pre-vllm-project#126 (translated):** - scripts/low_precision/run-qwen3-4b-fp8.sh - scripts/low_precision/run-qwen3-30b-a3b-fp8.sh - scripts/run-glm4-9B.sh - scripts/run-moonlight-16B-A3B.sh - scripts/run-qwen3-4B-base-sft.sh - scripts/run-qwen3-32B.sh - scripts/run-qwen3.5-35B-A3B-sft.sh **New from slime@44d29ee (translated):** - docs/en/get_started/agent.md - examples/fully_async/run-qwen2.5-0.5B-fully_async.sh All sglang engine flags translated to vllm equivalents (§2.4). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Standardize all scripts to use the bracket-escaped pkill pattern that avoids matching pkill itself and also catches vLLM's renamed subprocesses (VLLM::EngineCore, VLLM::Worker_TP*). Matches the canonical pattern in command_utils.py. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request adds and updates several training and rollout shell scripts for various models, including Qwen, Kimi-K2, DeepSeek-R1, and GLM, to support low-precision training (INT4 and FP8) and integrate vLLM. The review feedback highlights several critical issues, including a missing trailing backslash in run-kimi-k2-Instruct.sh that breaks the Ray job submission, incorrect relative source paths for model configurations across multiple scripts, leftover paths and package names from the 'slime' repository, a typo in the Python buffering environment variable, and a leading blank line before the shebang in run-mimo-7B-rl-eagle.sh.
| --actor-num-nodes 32 \ | ||
| --actor-num-gpus-per-node 8 \ | ||
| --colocate \ | ||
| --update-weight-buffer-size $(( 4 * 512 * 1024 * 1024)) |
Contributor
There was a problem hiding this comment.
The line is missing a trailing backslash \. This will cause the shell to treat the subsequent lines as a separate command, breaking the ray job submit execution.
Suggested change
| --update-weight-buffer-size $(( 4 * 512 * 1024 * 1024)) | |
| --update-weight-buffer-size $(( 4 * 512 * 1024 * 1024)) \ |
| # --global-batch-size 256 | ||
|
|
||
| --over-sampling-batch-size 256 | ||
| --dynamic-sampling-filter-path slime.rollout.filter_hub.dynamic_sampling_filters.check_reward_nonzero_std |
Contributor
There was a problem hiding this comment.
The package has been renamed/translated from slime to vime (as seen in the codebase structure, e.g., vime/rollout/vllm_rollout.py). Using slime.rollout... will result in a ModuleNotFoundError. Please update this path to use vime instead of slime.
Suggested change
| --dynamic-sampling-filter-path slime.rollout.filter_hub.dynamic_sampling_filters.check_reward_nonzero_std | |
| --dynamic-sampling-filter-path vime.rollout.filter_hub.dynamic_sampling_filters.check_reward_nonzero_std |
|
|
||
| ray job submit --address="http://127.0.0.1:8265" \ | ||
| --runtime-env-json="${RUNTIME_ENV_JSON}" \ | ||
| -- python3 /personal/slime/slime/train.py \ |
Contributor
There was a problem hiding this comment.
The script is executing /personal/slime/slime/train.py which is a leftover path from the slime repository. It should be updated to train.py to run the vime training script in the current workspace, consistent with the other run scripts.
Suggested change
| -- python3 /personal/slime/slime/train.py \ | |
| -- python3 train.py \ |
| echo "HAS_NVLINK: $HAS_NVLINK (detected $NVLINK_COUNT NVLink references)" | ||
|
|
||
| SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)" | ||
| source "${SCRIPT_DIR}/../scripts/models/qwen3-30B-A3B.sh" |
Contributor
There was a problem hiding this comment.
The source path ../scripts/models/qwen3-30B-A3B.sh is incorrect. Since this script is located in scripts/low_precision/, .. resolves to scripts/, making the path scripts/scripts/models/... which does not exist. It should be ../models/qwen3-30B-A3B.sh.
Suggested change
| source "${SCRIPT_DIR}/../scripts/models/qwen3-30B-A3B.sh" | |
| source "${SCRIPT_DIR}/../models/qwen3-30B-A3B.sh" |
| echo "HAS_NVLINK: $HAS_NVLINK (detected $NVLINK_COUNT NVLink references)" | ||
|
|
||
| SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)" | ||
| source "${SCRIPT_DIR}/../scripts/models/qwen3-4B.sh" |
Contributor
There was a problem hiding this comment.
The source path ../scripts/models/qwen3-4B.sh is incorrect. Since this script is located in scripts/low_precision/, .. resolves to scripts/, making the path scripts/scripts/models/... which does not exist. It should be ../models/qwen3-4B.sh.
Suggested change
| source "${SCRIPT_DIR}/../scripts/models/qwen3-4B.sh" | |
| source "${SCRIPT_DIR}/../models/qwen3-4B.sh" |
|
|
||
| SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)" | ||
| source "${SCRIPT_DIR}/models/qwen2.5-0.5B.sh" | ||
| source "${SCRIPT_DIR}/scripts/models/qwen2.5-0.5B.sh" |
Contributor
There was a problem hiding this comment.
The source path was incorrectly changed to ${SCRIPT_DIR}/scripts/models/.... Since this script is located in scripts/, ${SCRIPT_DIR} is already scripts/, making the path scripts/scripts/models/... which does not exist. It should be reverted to ${SCRIPT_DIR}/models/qwen2.5-0.5B.sh.
Suggested change
| source "${SCRIPT_DIR}/scripts/models/qwen2.5-0.5B.sh" | |
| source "${SCRIPT_DIR}/models/qwen2.5-0.5B.sh" |
| set -ex | ||
|
|
||
| export PYTHONUNBUFFERED=1 | ||
| export PYTHONBUFFERED=16 |
Contributor
There was a problem hiding this comment.
The environment variable PYTHONBUFFERED=16 is a typo. The standard Python environment variable to control buffering is PYTHONUNBUFFERED (typically set to 1 to disable buffering). Python does not recognize PYTHONBUFFERED.
Suggested change
| export PYTHONBUFFERED=16 | |
| export PYTHONUNBUFFERED=1 |
Comment on lines
+1
to
+2
|
|
||
| #!/bin/bash |
Contributor
There was a problem hiding this comment.
There is a leading empty line before the shebang #!/bin/bash. For a shebang to be recognized by the OS kernel when executing the script directly, it must be the absolute first line of the file.
Suggested change
| #!/bin/bash | |
| #!/bin/bash |
| # 229B MoE, 256 experts -> requires many GPUs | ||
| # Typical config: TP=2, PP=2, EP=4, training side 16 GPUs (2 nodes x 8 GPUs) | ||
| # Inference side: vLLM on separate GPUs, EP=16+ | ||
| # Inference side: SGLang on separate GPUs, EP=16+ |
Contributor
There was a problem hiding this comment.
The comment was updated to refer to SGLang instead of vLLM. Since this PR is migrating the codebase from SGLang to vLLM, this comment is backwards and misleading. It should refer to vLLM.
Suggested change
| # Inference side: SGLang on separate GPUs, EP=16+ | |
| # Inference side: vLLM on separate GPUs, EP=16+ |
aoshen02
force-pushed
the
scripts/gb300-complete-port
branch
2 times, most recently
from
June 16, 2026 14:50
d5c572e to
e5d6f3a
Compare
aoshen02
force-pushed
the
scripts/gb300-complete-port
branch
from
June 16, 2026 14:53
e5d6f3a to
2864b34
Compare
Translate all slime scripts to vime following SGLANG_TO_VLLM_TRANSLATION.md: - sglang→vllm prefix swap for CLI flags and variables - _slime→_vime for checkpoint paths - EP: --sglang-ep-size N → --vllm-enable-expert-parallel (boolean) - Speculative: multi-param → --vllm-speculative-config JSON (§5.2) - Delete genuinely sglang-coupled params (DP-attention, DeepEP, NSA, etc.) - flashinfer → FLASHINFER case fix (§2.4) 23 new scripts + 6 existing updated to match slime@cutoff. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…cripts
The FP8 scripts used `${SCRIPT_DIR}/../scripts/models/` which resolves
to `scripts/scripts/models/` (non-existent). Changed to `../models/`
to match the INT4 scripts. Same fix as slime PR #2094.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three fixes needed to run GPT-OSS 20B RLHF on vLLM backend:
1. hf_weight_iterator_bridge: match Megatron-Bridge 0.5.0 API
_patch_bridge_expert_cache_to_cpu monkey-patches GPTOSSBridge.
maybe_modify_converted_hf_weight gained a 4th `hf_state_dict`
parameter; the patched wrapper only accepted 3, causing TypeError
during weight sync.
2. run-gpt-oss-20B: point --hf-checkpoint at fused BF16 format
vLLM's _load_weights_other expects gate_up_proj [E, hidden, 2*ffn]
(fused). The old per-expert split format (experts.{e}.gate_proj.weight)
causes KeyError on bias loading. Use tools/convert_gpt_oss_to_fused.py
to convert an existing per-expert checkpoint, or re-run
preprocess_gpt_oss.py to produce fused format directly.
3. run-gpt-oss-20B: add --qkv-format bshd + fix seq-length
GPT-OSS uses learnable softmax (sink attention). TransformerEngine
disables all attention backends when softmax_type=learnable and
qkv_format=thd (packed sequences). --qkv-format bshd avoids this.
--use-dynamic-batch-size is incompatible with bshd; replaced with
fixed --seq-length 10240 (covers 8192 max response + prompt headroom).
tools/convert_gpt_oss_to_fused.py: new tool to convert per-expert BF16
checkpoint (output of old preprocess_gpt_oss.py) to the fused HF format
expected by vLLM without re-running the slow MXFP4 dequantization.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
pkill -9 vllm matches any process named "vllm" and can inadvertently kill unrelated vllm processes (e.g. background services). Use the same pattern as PR vllm-project#220 which targets only vllm serve and Ray VLL[M]:: actors: pkill -9 -f '[v]llm serve|VLL[M]::' Also updates the inline form used in multi-node SSH worker restart commands (run-qwen3-235B-A22B*.sh, run-qwen3.5-27B.sh, etc.). Skipped: scripts/run-gpt-oss-20B.sh (uses pkill -9 -f "vllm serve" already), scripts/run-minimax-m2.sh and run-glm4.7-*.sh (already used -f "vllm serve"). Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…b300-complete-port # Conflicts: # scripts/run-glm4-9B.sh # scripts/run-moonlight-16B-A3B.sh # scripts/run-qwen3-32B.sh # scripts/run-qwen3-4B-base-sft.sh # scripts/run-qwen3.5-35B-A3B-sft.sh
aoshen02
changed the title
Documentation build overview
33 files changed ·
|
AMD-specific script is out of scope for the gb300-complete-port PR. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
… translations
- Add missing EN/ZH docs: low-precision, on-policy-distillation, get_started/agent,
pd-disaggregation (heterogeneous server groups fix), examples zh docs
- Add missing examples: on_policy_distillation, eval_multi_task, delta_weight_sync,
geo3k images
- Fix vLLM flag translations across all example docs:
- --vllm-mem-fraction-static → --vllm-gpu-memory-utilization
- Remove non-existent dp-attention flags (--vllm-enable-dp-attention, --vllm-dp-size,
--vllm-moe-dense-tp-size, --vllm-enable-dp-lm-head, --vllm-ep-size)
- --vllm-ep-num-redundant-experts → --vllm-eplb-config
- --vllm-cuda-graph-bs → --vllm-max-cudagraph-capture-size
- sglang speculative flags → --vllm-speculative-config JSON
- GLM-4.7 MTP: method=eagle → method=mtp, num_speculative_tokens=4 → 3
- sgl-router → vllm-router; THUDM/vime → vllm-project/vime
- Fix scripts: restore run-kimi-k2-Instruct/Thinking/qwen3-4B/qwen3-235B-A22B to
slime-44d29ee-as-vime + pkill precision fix only; restore int4 python3 path
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…h slime Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
… vllm) Covers examples/, docs/, tests/, and vime/utils -- previously missed in the scripts/ revert. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
aoshen02
changed the title
CalvinXKY
reviewed
Jun 25, 2026
| @@ -0,0 +1,170 @@ | |||
| #!/bin/bash | |||
Collaborator
There was a problem hiding this comment.
There is also a GLM5.2 script; would it be possible to migrate that as well?
Collaborator
|
Have these models + scripts been tested? I'd suggest running them all with the agent, and then adding a checklist to the description. |
Collaborator
Author
I've tested 80%+ of them except for deepseek r1 and the largest glm model. |
CalvinXKY
approved these changes
Jun 25, 2026
aoshen02
added a commit
that referenced
this pull request
Jun 25, 2026
Rebased onto upstream/main 7198547 (now includes #260 scripts port, #280 tau-bench agent_strategy, #283 docker image rename, #257 data fix). Effect vs prior base: the 12 script files #260 ported moved from new-to-vime to in-scope; 11 auto-merged clean, only scripts/run-deepseek-r1.sh newly conflicts (theirs adds /sgl-workspace dead path -> resolve by dropping). 3-way merge results on new base: 52 clean / 46 conflict (110 blocks) / 50 new-to-vime / 5 del Resolves PR #286 conflict-with-main. Conflict markers preserved for commit 2. SOP: knowledge/rl/slime-to-vime-sync-sop.md
aoshen02
added a commit
that referenced
this pull request
Jun 25, 2026
Mechanical commit 1 of 2 (per knowledge/rl/slime-to-vime-sync-sop.md §2). diff3 translated 3-way merge on upstream/main (incl #260/#280/#283/#257): ours = vime@main, base = translate(slime@#2013), theirs = translate(slime@#2125) Translation fixes vs prior attempt: - casing: SGLang->vLLM (prose) / SGLang<X>->VLLM<X> (identifiers); killed VLlm artifact (was 35 files) - dotted module refs slime.X->vime.X now translated (was leaking 'from slime.backends') These resolved 9 spurious conflicts (46->37 files). Results: 61 clean / 37 conflict (diff3 markers preserved) / 50 new-to-vime / 5 del. Conflict markers use readable -L labels (ours/base/theirs). Resolve in commit 2. Non-conflict provenance fix: vimerl/vime -> vllm/vime in 2 example docs. Engine patch handling (docker/patch/) deferred to commit 2 per SOP §4.5.
aoshen02
added a commit
that referenced
this pull request
Jun 26, 2026
Mechanical commit 1 of 2 (per knowledge/rl/slime-to-vime-sync-sop.md §2). diff3 translated 3-way merge on upstream/main (incl #260/#280/#283/#257): ours = vime@main, base = translate(slime@#2013), theirs = translate(slime@#2125) Translation fixes vs prior attempt: - casing: SGLang->vLLM (prose) / SGLang<X>->VLLM<X> (identifiers); killed VLlm artifact (was 35 files) - dotted module refs slime.X->vime.X now translated (was leaking 'from slime.backends') These resolved 9 spurious conflicts (46->37 files). Results: 61 clean / 37 conflict (diff3 markers preserved) / 50 new-to-vime / 5 del. Conflict markers use readable -L labels (ours/base/theirs). Resolve in commit 2. Non-conflict provenance fix: vimerl/vime -> vllm/vime in 2 example docs. Engine patch handling (docker/patch/) deferred to commit 2 per SOP §4.5. Signed-off-by: aoshen02 <aoshen@inferact.ai>
aoshen02
added a commit
that referenced
this pull request
Jun 26, 2026
…y actors PR #260 ("complete slime-exact port") changed execute_train's pre-launch cleanup from pkill -9 -f '[v]llm serve|VLL[M]::' to pkill -9 vllm as an over-literal sglang->vllm translation. But the vLLM rollout engine runs as Ray actor processes whose process *name* is python/ray, with "VLLM::" only in the command line — so `pkill -9 vllm` (name match, no -f) does not kill them. Leftover engine processes from the ckpt test's save phase survive into the load phase, holding ~115 GiB, so the load-phase engine starts with ~24/139 GiB free and dies with "Free memory ... less than desired GPU memory utilization (0.8, 111.84 GiB)" (test_qwen3_4B_ckpt.py, both --async-save and not). Restore the cmdline-match pattern `-f '[v]llm serve|VLL[M]::'`. Bisected on h200: ckpt PASSES at 289ee6d / e62d44f (old pattern, 4/4 runs) and FAILS at 7198547/main + PR (new pattern, 0/3), same old image -> code regression in #260. Verified: pkill-fixed PR code + new pr286 image -> ckpt PASS (579s). Signed-off-by: aoshen02 <aoshen@inferact.ai>
CalvinXKY
pushed a commit
that referenced
this pull request
Jun 29, 2026
…eview] (#286) * sync(slime #2014..#2125): diff3 3-way merge, conflicts preserved Mechanical commit 1 of 2 (per knowledge/rl/slime-to-vime-sync-sop.md §2). diff3 translated 3-way merge on upstream/main (incl #260/#280/#283/#257): ours = vime@main, base = translate(slime@#2013), theirs = translate(slime@#2125) Translation fixes vs prior attempt: - casing: SGLang->vLLM (prose) / SGLang<X>->VLLM<X> (identifiers); killed VLlm artifact (was 35 files) - dotted module refs slime.X->vime.X now translated (was leaking 'from slime.backends') These resolved 9 spurious conflicts (46->37 files). Results: 61 clean / 37 conflict (diff3 markers preserved) / 50 new-to-vime / 5 del. Conflict markers use readable -L labels (ours/base/theirs). Resolve in commit 2. Non-conflict provenance fix: vimerl/vime -> vllm/vime in 2 example docs. Engine patch handling (docker/patch/) deferred to commit 2 per SOP §4.5. Signed-off-by: aoshen02 <aoshen@inferact.ai> * sync(slime #2014..#2125): resolve all conflicts (commit 2) Resolved all 37 conflict files / 84 diff3 blocks per agent_run RESOLUTION_POLICY. Principle: keep vime vLLM impl (ours) + incorporate slime's new features (theirs). Highlights: - vLLM API form kept everywhere: /inference/v1/generate, choices parsing, AsyncEngineArgs, vLLM flag names (--vllm-gpu-memory-utilization etc). - Dropped all vllm.srt.* imports (non-existent in real vLLM). - Accepted new slime features: delta-weight-sync CLI args, append_response_tokens (Sample), get_server_info/start_external_rollout_servers/get_rollout_num_engines, old-router(<=0.2.1) compat, TrajectoryManager adapter design (vime already adopted it). - Kept vime-only: --rollout-external, add_router_arguments, _get_metrics_router_addr, reinit_wandb_primary_with_open_metrics, update_tracking_open_metrics, modal sandbox, VIME_AGENT_* env names, local-vLLM tau-bench user sim. - Engine patches (docker/patch/): kept ours vllm.patch (22-line MoE fix), dropped theirs sglang 2674-line content; deleted sglang-only vllm-top_p.patch (per SOP 4.5). - Dockerfile kept ours (vllm/vllm-openai base); version.txt accepted theirs nightly. - run-deepseek-r1.sh: dropped /sgl-workspace dead-path env. Deviations from policy (documented): vllm_rollout.py abort path kept ours pause/drain (abort_servers_until_idle would break partial-rollout drain + leave paused_workers unbound). README ecosystem section left empty (ours) pending de-translation of provenance. All changed .py py_compile clean; zero conflict markers; no sglang/slime leakage. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): pre-commit lint/format on resolved files - re-add dropped `from vllm_router.launch_router import RouterArgs` import in vime/utils/arguments.py (add_router_arguments uses it; F821 from conflict resolution) - drop unused base_top_p_token_ids/offsets in vllm_streaming_rollout.py (F841; came from theirs but ours's choices-parsing path doesn't use them) - black/isort autoformat (anthropic.py, test_agent/*, arguments.py) - pipeline.yml: agent tests moved to tests/test_agent/; wire new CPU tests pre-commit: all hooks pass (ruff/autoflake/isort/black/yaml). Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): make CPU CI green (engine import, docker build, agent/utils tests) Local CPU CI (pre-commit + plugin + agent + utils) all green on 8xH200 host in python:3.11 containers. Fixes found while running it: - vllm_engine.py: make 'import vllm_router'/'packaging.parse' lazy (inside _register_to_router); top-level import broke CPU import (vllm_router absent in CPU CI). The old-router(<=0.2.1) compat branch is theirs-accepted. - docker/Dockerfile: TMS_CUDA_MAJOR=12 for torch_memory_saver pin (its build backend now requires it for CUDA wheels; base is cu129). Unblocks image build. - agent/adapters/common.py: _run_turn called parse_model_output() without the required tokenizer= kwarg -> 500s in adapter tests. Pass tokenizer=tok. - tests/test_agent/_fakes.py: FakeVLLMServer served sglang /generate + meta_info; retarget to vime /inference/v1/generate + choices shape + x-session-id header. - tests/test_agent/test_adapters.py: parse_model_output(tokenizer=...) + assert vime body keys (token_ids/max_tokens). - tests/utils/test_vllm_config.py: vLLMConfig->VllmConfig (4 sites); fake router returns 3-tuple (ip,port,prom) matching _start_router; drop spurious resolve(). - tests/test_megatron_argument_validation.py: add num_gpus_per_node=8 to the vime_validate_args fixture (vime colocate override needs it). - .buildkite/pipeline.yml: agent tests -> tests/test_agent/*; +cispo_loss, +logprob_response_spans (CPU-safe); test_rollout_metrics stays GPU-only (imports vllm). Engine patch verdict (PATCH_ASSESSMENT.md): P1-P7 vLLM doesn't need (NIXL/Mooncake native); kept ours vllm.patch, dropped sglang content + top_p.patch. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): unblock GPU run (scipy pin, router arg dup, top-p-replay gate) Found running GPU CI on 8xH200 with vllm/vime:latest: - docker/Dockerfile: pin scipy<1.14 next to numpy<2 (scipy drifted to 1.18 which needs numpy>=2 and uses removed np.long -> 'import vllm' crash). - vllm_utils/arguments.py: drop sync-added RouterArgs.add_cli_args + its import in add_vllm_router_arguments. main exposes the full router surface only in utils.add_router_arguments; the duplicate re-registered --router-request-timeout-secs -> argparse conflict at train startup. - megatron_utils/loss.py: get_rollout_top_p_logprob_kwargs falls back to full-vocab logprob when top-p nucleus token ids are absent instead of raising. slime's top-p-replay needs engine-returned top-p tokens; vime's vLLM /inference/v1/generate does not expose them (sglang-only). Matches vime pre-sync behavior; flagged in OVERNIGHT_REPORT for review. Image import smoke + Megatron ckpt load + 4x VLLMEngine bringup + NCCL weight transfer all confirmed working in-image before this. Signed-off-by: aoshen02 <aoshen@inferact.ai> * ci(gpu): drop deleted test_qwen2.5_0.5B_ppo_critic_only_short from short suite slime deleted tests/test_qwen2.5_0.5B_ppo_critic_only_short.py this window (#2014..#2125); gpu_suites.py still listed it -> 'no such file' exit 2. Critic-only path is still covered by test_qwen3_4B_ppo_train_critic_only (megatron suite). Other 3 short tests (gsm8k_async, gsm8k, fully_async) pass on 8xH200. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): restore base_log_probs init in streaming rollout (None+list crash) GPU test_qwen3_4B_streaming_partial_rollout hit 'TypeError: unsupported operand +: NoneType and list' at vllm_streaming_rollout.py:234. The conflict resolution changed base_log_probs from main's `list(sample.rollout_log_probs or [])` to a None-able form; a fresh sample (rollout_log_probs=None) then did None + call_log_probs. Restored main's form. Real sync-resolution regression caught by GPU CI. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): streaming rollout uses append_response_tokens (was renamed update_from_meta_info) slime #2110 renamed Sample.update_from_meta_info -> append_response_tokens; vllm_rollout was updated but vllm_streaming_rollout still called the old name (AttributeError at generate_streaming). Streaming already accumulates tokens incrementally for partial-rollout, so call append_response_tokens(meta_info=meta) with tokens omitted -> metadata-only finalize (no double-append). Caught by GPU test_qwen3_4B_streaming_partial_rollout. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): restore vime unconditional colocate rollout_num_gpus re-derive 3-way compare (slime / vime-main / PR) showed the merge made a broken hybrid: it KEPT vime's num_gpus_per_node colocate override (which assumes rollout_num_gpus is forced to actor_num_gpus_per_node*actor_num_nodes) but REPLACED vime's unconditional re-derive (`!= -> re-derive`) with slime's `is None`-only form. When a colocate test's rollout_num_gpus is non-None but mismatches, it was left mis-sized -> engine/GPU misplacement -> mixed_offload IPC-UUID mismatch + ckpt 'Free memory < util'. slime passes (no override, self-consistent is-None); vime main passes (override + unconditional re-derive, coupled). Restore vime's re-derive; keep slime's new rollout_num_gpus==0 branch (checked first). Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): slime-consistency review pass (docs, rollout routed_experts, comments) Docs (EN+zh parity): - fault-tolerance: revert junk 'server'->'engine' mistranslation (keep correct /health endpoint, verified vs vllm_engine.py) - vllm-config: 'ServerArgs'->'EngineArgs' (sglang class -> vLLM AsyncEngineArgs u FrontendArgs); fix duplicated 'vllm-router (vllm-router)' alias - customization: restore over-deleted 'custom_generate -> list[Sample]' section + signature (dropped only the vime-absent search-r1 example link) Rollout: - routed_experts now flows through the slime-identical Sample._apply_meta_info (single assignment site, torch.int32 tensor matching downstream) instead of an inline numpy assign; vLLM .npy-on-choice decode stays (engine wire-format delta). Both vllm_rollout and vllm_streaming_rollout. Comments for future syncers: - --opd-teacher-model + on_policy_distillation: engine-driven divergence (vLLM model field; sglang /generate has none) - overrides / _vllm_server_field_names: AsyncEngineArgs u FrontendArgs == slime's sglang ServerArgs Examples/docker/etc: drop vime-absent npu/retool/search-r1/tau-bench files; restore eval_multi_task; docker alignment with slime. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(ci): pre-commit green — define base in streaming MM render (F821) + isort/black on test_agent Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(ci): correct two mistranslated CPU tests (colocate rollout-gpu re-derive; parse_model_output tokenizer) - test_..._preserves_larger_rollout_gpus_under_colocate asserted slime behavior (==12); vime re-derives to actor*nodes=8 under colocate (commit 9701304). Renamed + assert ==8 + divergence note. vime-main never had the test; slime does. - test_parse_model_output_plain_text_no_parsers called parse_model_output without the required tokenizer kwarg (#198 made it required for vLLM parsers). Pass tokenizer=None (unused on the no-parser path). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): streaming rollout posts to /inference/v1/generate, not sglang /generate The slime diff3 merge took slime's sglang endpoint (/generate) for the streaming rollout POST instead of keeping vime's vLLM endpoint (/inference/v1/generate). main (7198547) had the correct URL; the sync regressed it (and dropped the base var). Result: 404 Not Found at vllm_streaming_rollout.py:182 -> test_qwen3_4B_streaming_partial_rollout fails. Caught on a clean h200 node. The file's own docstrings already say /inference/v1/generate throughout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): thread rollout port cursor globally across multi-model engines The diff3 merge adopted slime's deferred rollout-engine init (start_rollout_servers now returns pending_init_handles awaited by the caller instead of ray.get-ing each model's engines before the next). That broke an implicit invariant the per-model port_cursors reset relied on: in main, model 0's engines were fully bound before model 1 allocated ports, so the free-port bind-test in _allocate_rollout_engine_addr_and_ports_normal skipped model 0's ports. With deferred init, model 1 allocates while model 0 is unbound, the bind-test sees the base ports free, and a second model (e.g. mixed_offload's frozen "ref") lands on the same 15000-15003 as the actor. The actor's POST /update_weights to :15002 then hits the never-started ref engine -> vLLM 500 "start_weight_update must be called before update_weights" (test_vllm_config_mixed_offload[_ft]). Fix: initialize port_cursors once before the model loop so the per-node next-free cursor is monotonic across all models, keeping every engine's ports disjoint regardless of bind timing. Single-model behaviour is unchanged; the per-model reset only existed to scope cursors that are already node-keyed. Caught on h200 GPU CI (new nightly-dev-20260618a image, which added the start_weight_update-before-update_weights enforcement that exposed the collision). Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): restore robust pkill pattern so ckpt cleanup kills vLLM ray actors PR #260 ("complete slime-exact port") changed execute_train's pre-launch cleanup from pkill -9 -f '[v]llm serve|VLL[M]::' to pkill -9 vllm as an over-literal sglang->vllm translation. But the vLLM rollout engine runs as Ray actor processes whose process *name* is python/ray, with "VLLM::" only in the command line — so `pkill -9 vllm` (name match, no -f) does not kill them. Leftover engine processes from the ckpt test's save phase survive into the load phase, holding ~115 GiB, so the load-phase engine starts with ~24/139 GiB free and dies with "Free memory ... less than desired GPU memory utilization (0.8, 111.84 GiB)" (test_qwen3_4B_ckpt.py, both --async-save and not). Restore the cmdline-match pattern `-f '[v]llm serve|VLL[M]::'`. Bisected on h200: ckpt PASSES at 289ee6d / e62d44f (old pattern, 4/4 runs) and FAILS at 7198547/main + PR (new pattern, 0/3), same old image -> code regression in #260. Verified: pkill-fixed PR code + new pr286 image -> ckpt PASS (579s). Signed-off-by: aoshen02 <aoshen@inferact.ai> * chore(sync): replace all `pkill -9 vllm` with cmdline-match pattern Same root cause as 764e1e1 (command_utils.py): `pkill -9 vllm` matches by process *name*, but vLLM rollout engines run as Ray actor processes (python/ray named, "VLLM::" only in the command line), so the name match never kills them. Apply the robust cmdline pattern `pkill -9 -f '[v]llm serve|VLL[M]::'` everywhere the bare `pkill -9 vllm` cleanup was used across run/example scripts, so leftover engines don't squat GPUs across runs. No logic change beyond the kill pattern. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(docker): restore vLLM core.py partial-wake sleep-guard (#44483) Commit d41f0aa removed the core.py sleep-guard hunk from docker/patch/latest/vllm.patch on the assumption it was "already in v0.23.0". It is not: stock v0.23.0 `vllm/v1/engine/core.py` calls `resume_scheduler()` and `execute_dummy_batch()` even during a partial (weights-only) wake. So a colocate DP+EP pd_mooncake rollout, right after `POST /wake_up?tags=weights` (KV cache still released under level-2 sleep), has its DP busy-loop fire a decode-shaped dummy batch that touches freed KV -> the scheduler_metadata write in flashattn_mla.py:234 (MLA, glm4.7) and flash_attn.py:547 (FA3, qwen3.6) raises `CUDA error: invalid argument`. Restore the guard (`if not self.model_executor.is_sleeping` around resume_scheduler; `if not self.is_sleeping()` around execute_dummy_batch), keeping the all2all_utils weight-reload fix. This is the #173 sleep-guard patch re-expressed against v0.23.0 line numbers. Verified: git-apply --check clean against stock v0.23.0; both guards land; glm4.7/qwen3.6 pd_mooncake reproduced the crash without it (the 8-day-old image that still carried the guard passes both). Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(docker): drop scipy<1.14 pin, mirror slime numpy<2 only The scipy<1.14 pin (added in 0cda11c while chasing the pd_mooncake crash) was a red herring: the real cause was the dropped core.py partial-wake sleep-guard, now restored. slime-2125-as-vime pins only `numpy<2`; this restores that exact line. numpy 1.26.4 + scipy resolved naturally matches the working baseline. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(docker): keep FlashQLA install gated behind INSTALL_FLASHQLA=0 slime installs FlashQLA unconditionally, but it is sm90/Hopper-only. Restore vime's original gated form (default off; --qwen-gdn-backend fla elsewhere). CI build passes --build-arg INSTALL_FLASHQLA=1 to include it. Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs(vllm-config): fix inference-only FAQ — vime launches engines in-process The mechanical mirror of slime's answer steered users to vLLM's standalone `vllm serve` (slime's `launch_server` analog) for inference-only. That is misleading for vime: like slime, vime launches the vLLM engines in-process from `--vllm-config` (same in-process path as training), so a rollout-only run serves directly with no separate server process. Point standalone users to `--rollout-external-engine-addrs` instead. EN + ZH. Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs(debug): restore INT4 / Compressed-Tensors checkpoint section vime's debug.md was missing slime's "INT4 / Compressed-Tensors Quantization Checkpoint Issues" section (slime #1642) — dropped in an earlier sync, not present on main. Restore it (EN + ZH), translated sglang→vLLM / Megatron→vLLM. Covers the quantization_config.ignore list, all-zero MoE router weights (mlp.gate.weight) when mis-quantized, missing safetensors shards, and diagnosis via --check-weight-update-equal / --debug-rollout-only. Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs(vllm-config): use _run_vllm_server for inference-only FAQ Keep slime's wording; the only engine-coupled fix is the standalone launcher name. slime's `launch_server` is its in-process engine entry; vime's analog is `_run_vllm_server` (vllm_engine.py, launched via multiprocessing.Process), not the standalone `vllm serve` CLI. EN + ZH. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(docker): restore scipy pin (scipy<1.18) — vime base needs it Reverts the scipy-pin removal in b359b24, which was wrong: vime's vllm/vllm-openai base ships no scipy, so unpinned the build pulls scipy>=1.18, which hard-requires numpy>=2 and uses np.long (removed numpy>=1.24) -> crashes against the numpy<2 reinstall (Megatron needs numpy 1.x). slime's sglang base resolves scipy 1.17.1 natively (numpy-1.x compatible), so slime needs no pin; this is a base-image divergence, not a red herring. Pin boundary is 1.18 (slime runs 1.17.1), not the earlier 1.14 over-estimate. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(scripts): properly translate sglang args in glm5.2-744B + glm4.7-355B-delta These two were prefix-swapped (--sglang-X -> --vllm-X) without semantic mapping, leaving ~20 args that aren't vLLM AsyncEngineArgs (argparse would reject). Apply the knowledge/rl/sglang-to-vllm-translation.md §5.5 mappings: - dp-size->data-parallel-size, ep-size->enable-expert-parallel, max-running-requests-> max-num-seqs, cuda-graph-max-bs->max-cudagraph-capture-size - 5x/4x --speculative-* -> one --vllm-speculative-config JSON (§5.2) - DeepEP: per-group deepep_mode auto/low_latency -> all2all_backend deepep_high_throughput/ low_latency in the --vllm-config overrides (vLLM has no 'auto'; PD encodes it per-role) - watchdog-timeout -> env VLLM_ENGINE_ITERATION_TIMEOUT_S - drop sglang-only: dp-attention / dp-lm-head / moe-dense-tp / disable-overlap-schedule / NSA backends (vLLM selects DeepSeek sparse attn per model) / engine delta-receiver knobs - flag PD mooncake transport (-> --vllm-kv-transfer-config) as fabric-specific TODO These are 744B/355B scripts not runnable in CI — translations are SOP-mapped but hardware-unvalidated (flagged inline). Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(args): hard-guard unverified delta weight-sync mode --update-weight-mode=delta (PR #278 lineage) is not yet validated on vime+vLLM (vLLM exposes dense/sparse_flat only, not slime's gap-delta/ zstd encoding). Raise NotImplementedError at arg-validation so it fails fast at startup instead of crashing mid weight-sync. Downstream delta code is kept untouched; remove this raise once a real delta-load run passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs,test: correct rollout engine endpoint to /inference/v1/generate The agent rollout engine is reached at vime's vLLM ``/inference/v1/generate`` (see vllm_rollout.get_model_url default + common.call_vllm_generate), not the bare ``/generate`` of sglang. Fix the imprecise path in adapter/test docstrings and comments, and rewrite the vllm-config.md custom-rollout examples (en+zh): they were still sglang-shaped (``/generate`` path + ``{"text":..., "return_logprob": True}`` body). Use vime's real request schema instead -- ``{"model","token_ids", "sampling_params"}`` with ``max_tokens``/``logprobs``, ``prompt_logprobs`` for fixed-sequence scoring, and the ``choices[0]`` response shape. No code/logic change: comments, docstrings, and doc examples only. The Megatron training server's own ``/generate`` endpoint and the sglang citation in arguments.py are correct and left untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * style(args): collapse delta-guard message to one line (black) The delta-guard NotImplementedError message was split across two adjacent string literals; black on the CI (line-length 119) collapses/normalizes it. Make it a single clean literal so pre-commit is green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * test(args): assert delta weight-sync is guarded off (not per-condition) The hard delta guard (7bb19e6) raises NotImplementedError at the top of the delta branch, making the downstream colocate / unknown-transport rejections unreachable. Replace test_update_weight_delta_rejects_colocate and test_update_weight_delta_rejects_unknown_transport (whose ValueError paths no longer fire) with a single test_update_weight_delta_disabled that asserts the guard raises for any delta config. Breadcrumb left to restore the per-condition tests when delta is verified and the guard is lifted. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): restore dropped weight-sync metrics chain + delta dispatch (slime parity) A mirror cross-check of the delta-weight-sync surface (slime #1806/#1991) found the #2014..#2125 sync had silently dropped several slime-faithful pieces: * extra_metrics logging chain — slime threads weight-update metrics from the actor through log_perf_data -> log_perf_data_raw. vime dropped the param at all three layers, so weight-update metrics were never logged. Restored: train_metric_utils.log_perf_data_raw(extra_metrics=...), data.log_perf_data passthrough, and actor passing self.weight_updater.pop_metrics(). * pop_metrics on UpdateWeightFromDistributed — the default (non-colocate, nccl) weight_updater. slime gives all three updaters a pop_metrics() stub so the actor can call it uniformly; vime kept it on tensor/disk but dropped it on distributed, which would AttributeError once the actor calls it. Restored the ~5-line stub (delta-specific plumbing stays dropped — vime+vLLM has no DeltaSpec). * actor delta-mode dispatch branch — restores the elif selecting UpdateWeightFromDistributedDelta. Dead code behind the validation guard that rejects --update-weight-mode=delta, so vime mirrors slime with the guard as the single divergence. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * chore(sync): align comments to the mechanical mirror Comment-only pass; no behavior change. Aligns vime comments to what a faithful slime->vime translation would carry: * Strip vime-divergence rationale markers (delta guard, opd-teacher-model, top-p fallback, colocate/delta tests). The rationale belongs in the divergence manifest, not inline; the guarded code + self-explanatory NotImplementedError messages stand on their own. * De-verbose vLLM-specific comments to mirror scale: the router-args block, the AsyncEngineArgs u FrontendArgs docstring, and the MoE-replay / streaming-rollout blocks that slime does not carry at that length. * Restore slime-original comments the sync had dropped or naively translated, with judgment translation of sglang-specific terms: session_id routing ("vLLM router", not the mechanical "Model Gateway"), "Prepare payload for vLLM server", the unique-session_id loop, and the pending-tasks wait. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs(delta): note delta weight-sync not yet verified on vime+vLLM (PR #286 review) Per review on PR #286: delta weight sync is documented here but the arg guard disables --update-weight-mode=delta. Add a top-of-page note (en + zh) so users see it before hitting NotImplementedError at argparse. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> --------- Signed-off-by: aoshen02 <aoshen@inferact.ai> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
aoshen02
added a commit
to aoshen02/vime
that referenced
this pull request
Jul 15, 2026
… 20B support (vllm-project#260) * restore: bring back deleted examples, scripts, and agent doc Restore files that were either deleted by vllm-project#126 ("trim examples to qwen3 only") or never synced from slime: **Reverted from pre-vllm-project#126 (translated):** - scripts/low_precision/run-qwen3-4b-fp8.sh - scripts/low_precision/run-qwen3-30b-a3b-fp8.sh - scripts/run-glm4-9B.sh - scripts/run-moonlight-16B-A3B.sh - scripts/run-qwen3-4B-base-sft.sh - scripts/run-qwen3-32B.sh - scripts/run-qwen3.5-35B-A3B-sft.sh **New from slime@44d29ee (translated):** - docs/en/get_started/agent.md - examples/fully_async/run-qwen2.5-0.5B-fully_async.sh All sglang engine flags translated to vllm equivalents (§2.4). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: unify pkill pattern to '[v]llm serve|VLL[M]::' Standardize all scripts to use the bracket-escaped pkill pattern that avoids matching pkill itself and also catches vLLM's renamed subprocesses (VLLM::EngineCore, VLLM::Worker_TP*). Matches the canonical pattern in command_utils.py. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * scripts: complete slime-exact translation of all 29 run scripts Translate all slime scripts to vime following SGLANG_TO_VLLM_TRANSLATION.md: - sglang→vllm prefix swap for CLI flags and variables - _slime→_vime for checkpoint paths - EP: --sglang-ep-size N → --vllm-enable-expert-parallel (boolean) - Speculative: multi-param → --vllm-speculative-config JSON (§5.2) - Delete genuinely sglang-coupled params (DP-attention, DeepEP, NSA, etc.) - flashinfer → FLASHINFER case fix (§2.4) 23 new scripts + 6 existing updated to match slime@cutoff. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(scripts): correct model config source path in FP8 low_precision scripts The FP8 scripts used `${SCRIPT_DIR}/../scripts/models/` which resolves to `scripts/scripts/models/` (non-existent). Changed to `../models/` to match the INT4 scripts. Same fix as slime PR #2094. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(gpt-oss): fused BF16 format, bridge API patch, bshd qkv format Three fixes needed to run GPT-OSS 20B RLHF on vLLM backend: 1. hf_weight_iterator_bridge: match Megatron-Bridge 0.5.0 API _patch_bridge_expert_cache_to_cpu monkey-patches GPTOSSBridge. maybe_modify_converted_hf_weight gained a 4th `hf_state_dict` parameter; the patched wrapper only accepted 3, causing TypeError during weight sync. 2. run-gpt-oss-20B: point --hf-checkpoint at fused BF16 format vLLM's _load_weights_other expects gate_up_proj [E, hidden, 2*ffn] (fused). The old per-expert split format (experts.{e}.gate_proj.weight) causes KeyError on bias loading. Use tools/convert_gpt_oss_to_fused.py to convert an existing per-expert checkpoint, or re-run preprocess_gpt_oss.py to produce fused format directly. 3. run-gpt-oss-20B: add --qkv-format bshd + fix seq-length GPT-OSS uses learnable softmax (sink attention). TransformerEngine disables all attention backends when softmax_type=learnable and qkv_format=thd (packed sequences). --qkv-format bshd avoids this. --use-dynamic-batch-size is incompatible with bshd; replaced with fixed --seq-length 10240 (covers 8192 max response + prompt headroom). tools/convert_gpt_oss_to_fused.py: new tool to convert per-expert BF16 checkpoint (output of old preprocess_gpt_oss.py) to the fused HF format expected by vLLM without re-running the slow MXFP4 dequantization. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(scripts): replace pkill -9 vllm with precise -f pattern (21 files) pkill -9 vllm matches any process named "vllm" and can inadvertently kill unrelated vllm processes (e.g. background services). Use the same pattern as PR vllm-project#220 which targets only vllm serve and Ray VLL[M]:: actors: pkill -9 -f '[v]llm serve|VLL[M]::' Also updates the inline form used in multi-node SSH worker restart commands (run-qwen3-235B-A22B*.sh, run-qwen3.5-27B.sh, etc.). Skipped: scripts/run-gpt-oss-20B.sh (uses pkill -9 -f "vllm serve" already), scripts/run-minimax-m2.sh and run-glm4.7-*.sh (already used -f "vllm serve"). Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * chore(scripts): remove run-qwen3-4B-amd.sh from this PR AMD-specific script is out of scope for the gb300-complete-port PR. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * sync(docs+scripts): port docs/examples from slime-44d29ee, fix script translations - Add missing EN/ZH docs: low-precision, on-policy-distillation, get_started/agent, pd-disaggregation (heterogeneous server groups fix), examples zh docs - Add missing examples: on_policy_distillation, eval_multi_task, delta_weight_sync, geo3k images - Fix vLLM flag translations across all example docs: - --vllm-mem-fraction-static → --vllm-gpu-memory-utilization - Remove non-existent dp-attention flags (--vllm-enable-dp-attention, --vllm-dp-size, --vllm-moe-dense-tp-size, --vllm-enable-dp-lm-head, --vllm-ep-size) - --vllm-ep-num-redundant-experts → --vllm-eplb-config - --vllm-cuda-graph-bs → --vllm-max-cudagraph-capture-size - sglang speculative flags → --vllm-speculative-config JSON - GLM-4.7 MTP: method=eagle → method=mtp, num_speculative_tokens=4 → 3 - sgl-router → vllm-router; THUDM/vime → vllm-project/vime - Fix scripts: restore run-kimi-k2-Instruct/Thinking/qwen3-4B/qwen3-235B-A22B to slime-44d29ee-as-vime + pkill precision fix only; restore int4 python3 path Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * revert(scripts): pkill -9 -f pattern back to pkill -9 vllm, align with slime Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * revert(pkill): align all remaining kill patterns with slime (pkill -9 vllm) Covers examples/, docs/, tests/, and vime/utils -- previously missed in the scripts/ revert. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * chore: remove gpt-oss-20B script and convert tool (moved to separate PR) Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
aoshen02
added a commit
to aoshen02/vime
that referenced
this pull request
Jul 15, 2026
…eview] (vllm-project#286) * sync(slime #2014..#2125): diff3 3-way merge, conflicts preserved Mechanical commit 1 of 2 (per knowledge/rl/slime-to-vime-sync-sop.md §2). diff3 translated 3-way merge on upstream/main (incl vllm-project#260/vllm-project#280/vllm-project#283/vllm-project#257): ours = vime@main, base = translate(slime@#2013), theirs = translate(slime@#2125) Translation fixes vs prior attempt: - casing: SGLang->vLLM (prose) / SGLang<X>->VLLM<X> (identifiers); killed VLlm artifact (was 35 files) - dotted module refs slime.X->vime.X now translated (was leaking 'from slime.backends') These resolved 9 spurious conflicts (46->37 files). Results: 61 clean / 37 conflict (diff3 markers preserved) / 50 new-to-vime / 5 del. Conflict markers use readable -L labels (ours/base/theirs). Resolve in commit 2. Non-conflict provenance fix: vimerl/vime -> vllm/vime in 2 example docs. Engine patch handling (docker/patch/) deferred to commit 2 per SOP §4.5. Signed-off-by: aoshen02 <aoshen@inferact.ai> * sync(slime #2014..#2125): resolve all conflicts (commit 2) Resolved all 37 conflict files / 84 diff3 blocks per agent_run RESOLUTION_POLICY. Principle: keep vime vLLM impl (ours) + incorporate slime's new features (theirs). Highlights: - vLLM API form kept everywhere: /inference/v1/generate, choices parsing, AsyncEngineArgs, vLLM flag names (--vllm-gpu-memory-utilization etc). - Dropped all vllm.srt.* imports (non-existent in real vLLM). - Accepted new slime features: delta-weight-sync CLI args, append_response_tokens (Sample), get_server_info/start_external_rollout_servers/get_rollout_num_engines, old-router(<=0.2.1) compat, TrajectoryManager adapter design (vime already adopted it). - Kept vime-only: --rollout-external, add_router_arguments, _get_metrics_router_addr, reinit_wandb_primary_with_open_metrics, update_tracking_open_metrics, modal sandbox, VIME_AGENT_* env names, local-vLLM tau-bench user sim. - Engine patches (docker/patch/): kept ours vllm.patch (22-line MoE fix), dropped theirs sglang 2674-line content; deleted sglang-only vllm-top_p.patch (per SOP 4.5). - Dockerfile kept ours (vllm/vllm-openai base); version.txt accepted theirs nightly. - run-deepseek-r1.sh: dropped /sgl-workspace dead-path env. Deviations from policy (documented): vllm_rollout.py abort path kept ours pause/drain (abort_servers_until_idle would break partial-rollout drain + leave paused_workers unbound). README ecosystem section left empty (ours) pending de-translation of provenance. All changed .py py_compile clean; zero conflict markers; no sglang/slime leakage. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): pre-commit lint/format on resolved files - re-add dropped `from vllm_router.launch_router import RouterArgs` import in vime/utils/arguments.py (add_router_arguments uses it; F821 from conflict resolution) - drop unused base_top_p_token_ids/offsets in vllm_streaming_rollout.py (F841; came from theirs but ours's choices-parsing path doesn't use them) - black/isort autoformat (anthropic.py, test_agent/*, arguments.py) - pipeline.yml: agent tests moved to tests/test_agent/; wire new CPU tests pre-commit: all hooks pass (ruff/autoflake/isort/black/yaml). Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): make CPU CI green (engine import, docker build, agent/utils tests) Local CPU CI (pre-commit + plugin + agent + utils) all green on 8xH200 host in python:3.11 containers. Fixes found while running it: - vllm_engine.py: make 'import vllm_router'/'packaging.parse' lazy (inside _register_to_router); top-level import broke CPU import (vllm_router absent in CPU CI). The old-router(<=0.2.1) compat branch is theirs-accepted. - docker/Dockerfile: TMS_CUDA_MAJOR=12 for torch_memory_saver pin (its build backend now requires it for CUDA wheels; base is cu129). Unblocks image build. - agent/adapters/common.py: _run_turn called parse_model_output() without the required tokenizer= kwarg -> 500s in adapter tests. Pass tokenizer=tok. - tests/test_agent/_fakes.py: FakeVLLMServer served sglang /generate + meta_info; retarget to vime /inference/v1/generate + choices shape + x-session-id header. - tests/test_agent/test_adapters.py: parse_model_output(tokenizer=...) + assert vime body keys (token_ids/max_tokens). - tests/utils/test_vllm_config.py: vLLMConfig->VllmConfig (4 sites); fake router returns 3-tuple (ip,port,prom) matching _start_router; drop spurious resolve(). - tests/test_megatron_argument_validation.py: add num_gpus_per_node=8 to the vime_validate_args fixture (vime colocate override needs it). - .buildkite/pipeline.yml: agent tests -> tests/test_agent/*; +cispo_loss, +logprob_response_spans (CPU-safe); test_rollout_metrics stays GPU-only (imports vllm). Engine patch verdict (PATCH_ASSESSMENT.md): P1-P7 vLLM doesn't need (NIXL/Mooncake native); kept ours vllm.patch, dropped sglang content + top_p.patch. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): unblock GPU run (scipy pin, router arg dup, top-p-replay gate) Found running GPU CI on 8xH200 with vllm/vime:latest: - docker/Dockerfile: pin scipy<1.14 next to numpy<2 (scipy drifted to 1.18 which needs numpy>=2 and uses removed np.long -> 'import vllm' crash). - vllm_utils/arguments.py: drop sync-added RouterArgs.add_cli_args + its import in add_vllm_router_arguments. main exposes the full router surface only in utils.add_router_arguments; the duplicate re-registered --router-request-timeout-secs -> argparse conflict at train startup. - megatron_utils/loss.py: get_rollout_top_p_logprob_kwargs falls back to full-vocab logprob when top-p nucleus token ids are absent instead of raising. slime's top-p-replay needs engine-returned top-p tokens; vime's vLLM /inference/v1/generate does not expose them (sglang-only). Matches vime pre-sync behavior; flagged in OVERNIGHT_REPORT for review. Image import smoke + Megatron ckpt load + 4x VLLMEngine bringup + NCCL weight transfer all confirmed working in-image before this. Signed-off-by: aoshen02 <aoshen@inferact.ai> * ci(gpu): drop deleted test_qwen2.5_0.5B_ppo_critic_only_short from short suite slime deleted tests/test_qwen2.5_0.5B_ppo_critic_only_short.py this window (#2014..#2125); gpu_suites.py still listed it -> 'no such file' exit 2. Critic-only path is still covered by test_qwen3_4B_ppo_train_critic_only (megatron suite). Other 3 short tests (gsm8k_async, gsm8k, fully_async) pass on 8xH200. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): restore base_log_probs init in streaming rollout (None+list crash) GPU test_qwen3_4B_streaming_partial_rollout hit 'TypeError: unsupported operand +: NoneType and list' at vllm_streaming_rollout.py:234. The conflict resolution changed base_log_probs from main's `list(sample.rollout_log_probs or [])` to a None-able form; a fresh sample (rollout_log_probs=None) then did None + call_log_probs. Restored main's form. Real sync-resolution regression caught by GPU CI. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): streaming rollout uses append_response_tokens (was renamed update_from_meta_info) slime #2110 renamed Sample.update_from_meta_info -> append_response_tokens; vllm_rollout was updated but vllm_streaming_rollout still called the old name (AttributeError at generate_streaming). Streaming already accumulates tokens incrementally for partial-rollout, so call append_response_tokens(meta_info=meta) with tokens omitted -> metadata-only finalize (no double-append). Caught by GPU test_qwen3_4B_streaming_partial_rollout. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): restore vime unconditional colocate rollout_num_gpus re-derive 3-way compare (slime / vime-main / PR) showed the merge made a broken hybrid: it KEPT vime's num_gpus_per_node colocate override (which assumes rollout_num_gpus is forced to actor_num_gpus_per_node*actor_num_nodes) but REPLACED vime's unconditional re-derive (`!= -> re-derive`) with slime's `is None`-only form. When a colocate test's rollout_num_gpus is non-None but mismatches, it was left mis-sized -> engine/GPU misplacement -> mixed_offload IPC-UUID mismatch + ckpt 'Free memory < util'. slime passes (no override, self-consistent is-None); vime main passes (override + unconditional re-derive, coupled). Restore vime's re-derive; keep slime's new rollout_num_gpus==0 branch (checked first). Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): slime-consistency review pass (docs, rollout routed_experts, comments) Docs (EN+zh parity): - fault-tolerance: revert junk 'server'->'engine' mistranslation (keep correct /health endpoint, verified vs vllm_engine.py) - vllm-config: 'ServerArgs'->'EngineArgs' (sglang class -> vLLM AsyncEngineArgs u FrontendArgs); fix duplicated 'vllm-router (vllm-router)' alias - customization: restore over-deleted 'custom_generate -> list[Sample]' section + signature (dropped only the vime-absent search-r1 example link) Rollout: - routed_experts now flows through the slime-identical Sample._apply_meta_info (single assignment site, torch.int32 tensor matching downstream) instead of an inline numpy assign; vLLM .npy-on-choice decode stays (engine wire-format delta). Both vllm_rollout and vllm_streaming_rollout. Comments for future syncers: - --opd-teacher-model + on_policy_distillation: engine-driven divergence (vLLM model field; sglang /generate has none) - overrides / _vllm_server_field_names: AsyncEngineArgs u FrontendArgs == slime's sglang ServerArgs Examples/docker/etc: drop vime-absent npu/retool/search-r1/tau-bench files; restore eval_multi_task; docker alignment with slime. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(ci): pre-commit green — define base in streaming MM render (F821) + isort/black on test_agent Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(ci): correct two mistranslated CPU tests (colocate rollout-gpu re-derive; parse_model_output tokenizer) - test_..._preserves_larger_rollout_gpus_under_colocate asserted slime behavior (==12); vime re-derives to actor*nodes=8 under colocate (commit 9701304). Renamed + assert ==8 + divergence note. vime-main never had the test; slime does. - test_parse_model_output_plain_text_no_parsers called parse_model_output without the required tokenizer kwarg (vllm-project#198 made it required for vLLM parsers). Pass tokenizer=None (unused on the no-parser path). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): streaming rollout posts to /inference/v1/generate, not sglang /generate The slime diff3 merge took slime's sglang endpoint (/generate) for the streaming rollout POST instead of keeping vime's vLLM endpoint (/inference/v1/generate). main (7198547) had the correct URL; the sync regressed it (and dropped the base var). Result: 404 Not Found at vllm_streaming_rollout.py:182 -> test_qwen3_4B_streaming_partial_rollout fails. Caught on a clean h200 node. The file's own docstrings already say /inference/v1/generate throughout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): thread rollout port cursor globally across multi-model engines The diff3 merge adopted slime's deferred rollout-engine init (start_rollout_servers now returns pending_init_handles awaited by the caller instead of ray.get-ing each model's engines before the next). That broke an implicit invariant the per-model port_cursors reset relied on: in main, model 0's engines were fully bound before model 1 allocated ports, so the free-port bind-test in _allocate_rollout_engine_addr_and_ports_normal skipped model 0's ports. With deferred init, model 1 allocates while model 0 is unbound, the bind-test sees the base ports free, and a second model (e.g. mixed_offload's frozen "ref") lands on the same 15000-15003 as the actor. The actor's POST /update_weights to :15002 then hits the never-started ref engine -> vLLM 500 "start_weight_update must be called before update_weights" (test_vllm_config_mixed_offload[_ft]). Fix: initialize port_cursors once before the model loop so the per-node next-free cursor is monotonic across all models, keeping every engine's ports disjoint regardless of bind timing. Single-model behaviour is unchanged; the per-model reset only existed to scope cursors that are already node-keyed. Caught on h200 GPU CI (new nightly-dev-20260618a image, which added the start_weight_update-before-update_weights enforcement that exposed the collision). Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): restore robust pkill pattern so ckpt cleanup kills vLLM ray actors PR vllm-project#260 ("complete slime-exact port") changed execute_train's pre-launch cleanup from pkill -9 -f '[v]llm serve|VLL[M]::' to pkill -9 vllm as an over-literal sglang->vllm translation. But the vLLM rollout engine runs as Ray actor processes whose process *name* is python/ray, with "VLLM::" only in the command line — so `pkill -9 vllm` (name match, no -f) does not kill them. Leftover engine processes from the ckpt test's save phase survive into the load phase, holding ~115 GiB, so the load-phase engine starts with ~24/139 GiB free and dies with "Free memory ... less than desired GPU memory utilization (0.8, 111.84 GiB)" (test_qwen3_4B_ckpt.py, both --async-save and not). Restore the cmdline-match pattern `-f '[v]llm serve|VLL[M]::'`. Bisected on h200: ckpt PASSES at 289ee6d / e62d44f (old pattern, 4/4 runs) and FAILS at 7198547/main + PR (new pattern, 0/3), same old image -> code regression in vllm-project#260. Verified: pkill-fixed PR code + new pr286 image -> ckpt PASS (579s). Signed-off-by: aoshen02 <aoshen@inferact.ai> * chore(sync): replace all `pkill -9 vllm` with cmdline-match pattern Same root cause as 764e1e1 (command_utils.py): `pkill -9 vllm` matches by process *name*, but vLLM rollout engines run as Ray actor processes (python/ray named, "VLLM::" only in the command line), so the name match never kills them. Apply the robust cmdline pattern `pkill -9 -f '[v]llm serve|VLL[M]::'` everywhere the bare `pkill -9 vllm` cleanup was used across run/example scripts, so leftover engines don't squat GPUs across runs. No logic change beyond the kill pattern. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(docker): restore vLLM core.py partial-wake sleep-guard (#44483) Commit d41f0aa removed the core.py sleep-guard hunk from docker/patch/latest/vllm.patch on the assumption it was "already in v0.23.0". It is not: stock v0.23.0 `vllm/v1/engine/core.py` calls `resume_scheduler()` and `execute_dummy_batch()` even during a partial (weights-only) wake. So a colocate DP+EP pd_mooncake rollout, right after `POST /wake_up?tags=weights` (KV cache still released under level-2 sleep), has its DP busy-loop fire a decode-shaped dummy batch that touches freed KV -> the scheduler_metadata write in flashattn_mla.py:234 (MLA, glm4.7) and flash_attn.py:547 (FA3, qwen3.6) raises `CUDA error: invalid argument`. Restore the guard (`if not self.model_executor.is_sleeping` around resume_scheduler; `if not self.is_sleeping()` around execute_dummy_batch), keeping the all2all_utils weight-reload fix. This is the vllm-project#173 sleep-guard patch re-expressed against v0.23.0 line numbers. Verified: git-apply --check clean against stock v0.23.0; both guards land; glm4.7/qwen3.6 pd_mooncake reproduced the crash without it (the 8-day-old image that still carried the guard passes both). Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(docker): drop scipy<1.14 pin, mirror slime numpy<2 only The scipy<1.14 pin (added in 0cda11c while chasing the pd_mooncake crash) was a red herring: the real cause was the dropped core.py partial-wake sleep-guard, now restored. slime-2125-as-vime pins only `numpy<2`; this restores that exact line. numpy 1.26.4 + scipy resolved naturally matches the working baseline. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(docker): keep FlashQLA install gated behind INSTALL_FLASHQLA=0 slime installs FlashQLA unconditionally, but it is sm90/Hopper-only. Restore vime's original gated form (default off; --qwen-gdn-backend fla elsewhere). CI build passes --build-arg INSTALL_FLASHQLA=1 to include it. Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs(vllm-config): fix inference-only FAQ — vime launches engines in-process The mechanical mirror of slime's answer steered users to vLLM's standalone `vllm serve` (slime's `launch_server` analog) for inference-only. That is misleading for vime: like slime, vime launches the vLLM engines in-process from `--vllm-config` (same in-process path as training), so a rollout-only run serves directly with no separate server process. Point standalone users to `--rollout-external-engine-addrs` instead. EN + ZH. Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs(debug): restore INT4 / Compressed-Tensors checkpoint section vime's debug.md was missing slime's "INT4 / Compressed-Tensors Quantization Checkpoint Issues" section (slime #1642) — dropped in an earlier sync, not present on main. Restore it (EN + ZH), translated sglang→vLLM / Megatron→vLLM. Covers the quantization_config.ignore list, all-zero MoE router weights (mlp.gate.weight) when mis-quantized, missing safetensors shards, and diagnosis via --check-weight-update-equal / --debug-rollout-only. Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs(vllm-config): use _run_vllm_server for inference-only FAQ Keep slime's wording; the only engine-coupled fix is the standalone launcher name. slime's `launch_server` is its in-process engine entry; vime's analog is `_run_vllm_server` (vllm_engine.py, launched via multiprocessing.Process), not the standalone `vllm serve` CLI. EN + ZH. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(docker): restore scipy pin (scipy<1.18) — vime base needs it Reverts the scipy-pin removal in b359b24, which was wrong: vime's vllm/vllm-openai base ships no scipy, so unpinned the build pulls scipy>=1.18, which hard-requires numpy>=2 and uses np.long (removed numpy>=1.24) -> crashes against the numpy<2 reinstall (Megatron needs numpy 1.x). slime's sglang base resolves scipy 1.17.1 natively (numpy-1.x compatible), so slime needs no pin; this is a base-image divergence, not a red herring. Pin boundary is 1.18 (slime runs 1.17.1), not the earlier 1.14 over-estimate. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(scripts): properly translate sglang args in glm5.2-744B + glm4.7-355B-delta These two were prefix-swapped (--sglang-X -> --vllm-X) without semantic mapping, leaving ~20 args that aren't vLLM AsyncEngineArgs (argparse would reject). Apply the knowledge/rl/sglang-to-vllm-translation.md §5.5 mappings: - dp-size->data-parallel-size, ep-size->enable-expert-parallel, max-running-requests-> max-num-seqs, cuda-graph-max-bs->max-cudagraph-capture-size - 5x/4x --speculative-* -> one --vllm-speculative-config JSON (§5.2) - DeepEP: per-group deepep_mode auto/low_latency -> all2all_backend deepep_high_throughput/ low_latency in the --vllm-config overrides (vLLM has no 'auto'; PD encodes it per-role) - watchdog-timeout -> env VLLM_ENGINE_ITERATION_TIMEOUT_S - drop sglang-only: dp-attention / dp-lm-head / moe-dense-tp / disable-overlap-schedule / NSA backends (vLLM selects DeepSeek sparse attn per model) / engine delta-receiver knobs - flag PD mooncake transport (-> --vllm-kv-transfer-config) as fabric-specific TODO These are 744B/355B scripts not runnable in CI — translations are SOP-mapped but hardware-unvalidated (flagged inline). Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(args): hard-guard unverified delta weight-sync mode --update-weight-mode=delta (PR vllm-project#278 lineage) is not yet validated on vime+vLLM (vLLM exposes dense/sparse_flat only, not slime's gap-delta/ zstd encoding). Raise NotImplementedError at arg-validation so it fails fast at startup instead of crashing mid weight-sync. Downstream delta code is kept untouched; remove this raise once a real delta-load run passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs,test: correct rollout engine endpoint to /inference/v1/generate The agent rollout engine is reached at vime's vLLM ``/inference/v1/generate`` (see vllm_rollout.get_model_url default + common.call_vllm_generate), not the bare ``/generate`` of sglang. Fix the imprecise path in adapter/test docstrings and comments, and rewrite the vllm-config.md custom-rollout examples (en+zh): they were still sglang-shaped (``/generate`` path + ``{"text":..., "return_logprob": True}`` body). Use vime's real request schema instead -- ``{"model","token_ids", "sampling_params"}`` with ``max_tokens``/``logprobs``, ``prompt_logprobs`` for fixed-sequence scoring, and the ``choices[0]`` response shape. No code/logic change: comments, docstrings, and doc examples only. The Megatron training server's own ``/generate`` endpoint and the sglang citation in arguments.py are correct and left untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * style(args): collapse delta-guard message to one line (black) The delta-guard NotImplementedError message was split across two adjacent string literals; black on the CI (line-length 119) collapses/normalizes it. Make it a single clean literal so pre-commit is green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * test(args): assert delta weight-sync is guarded off (not per-condition) The hard delta guard (7bb19e6) raises NotImplementedError at the top of the delta branch, making the downstream colocate / unknown-transport rejections unreachable. Replace test_update_weight_delta_rejects_colocate and test_update_weight_delta_rejects_unknown_transport (whose ValueError paths no longer fire) with a single test_update_weight_delta_disabled that asserts the guard raises for any delta config. Breadcrumb left to restore the per-condition tests when delta is verified and the guard is lifted. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): restore dropped weight-sync metrics chain + delta dispatch (slime parity) A mirror cross-check of the delta-weight-sync surface (slime #1806/#1991) found the #2014..#2125 sync had silently dropped several slime-faithful pieces: * extra_metrics logging chain — slime threads weight-update metrics from the actor through log_perf_data -> log_perf_data_raw. vime dropped the param at all three layers, so weight-update metrics were never logged. Restored: train_metric_utils.log_perf_data_raw(extra_metrics=...), data.log_perf_data passthrough, and actor passing self.weight_updater.pop_metrics(). * pop_metrics on UpdateWeightFromDistributed — the default (non-colocate, nccl) weight_updater. slime gives all three updaters a pop_metrics() stub so the actor can call it uniformly; vime kept it on tensor/disk but dropped it on distributed, which would AttributeError once the actor calls it. Restored the ~5-line stub (delta-specific plumbing stays dropped — vime+vLLM has no DeltaSpec). * actor delta-mode dispatch branch — restores the elif selecting UpdateWeightFromDistributedDelta. Dead code behind the validation guard that rejects --update-weight-mode=delta, so vime mirrors slime with the guard as the single divergence. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * chore(sync): align comments to the mechanical mirror Comment-only pass; no behavior change. Aligns vime comments to what a faithful slime->vime translation would carry: * Strip vime-divergence rationale markers (delta guard, opd-teacher-model, top-p fallback, colocate/delta tests). The rationale belongs in the divergence manifest, not inline; the guarded code + self-explanatory NotImplementedError messages stand on their own. * De-verbose vLLM-specific comments to mirror scale: the router-args block, the AsyncEngineArgs u FrontendArgs docstring, and the MoE-replay / streaming-rollout blocks that slime does not carry at that length. * Restore slime-original comments the sync had dropped or naively translated, with judgment translation of sglang-specific terms: session_id routing ("vLLM router", not the mechanical "Model Gateway"), "Prepare payload for vLLM server", the unique-session_id loop, and the pending-tasks wait. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs(delta): note delta weight-sync not yet verified on vime+vLLM (PR vllm-project#286 review) Per review on PR vllm-project#286: delta weight sync is documented here but the arg guard disables --update-weight-mode=delta. Add a top-of-page note (en + zh) so users see it before hitting NotImplementedError at argparse. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> --------- Signed-off-by: aoshen02 <aoshen@inferact.ai> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR consolidates three work streams:
1. slime-exact translation of run scripts (original scope)
_slime→_vimecheckpoint paths, EP boolean conversion, speculative config merge to JSONSGLANG_TO_VLLM_TRANSLATION.md2. Restore deleted examples and scripts (from PR #220)
examples/coding_agent_rl/,examples/geo3k_vlm/,examples/multi_agent/,examples/train_infer_mismatch_helper/scripts/run-glm4.7-30B-A3B.sh,run-glm4.7-355B-A32B.sh,run-minimax-m2.sh,run-qwen3-30B-A3B.sh🤖 Generated with Claude Code