chore: trim examples to qwen3 only - #126
Conversation
caf0638 to
684be18
Compare
There was a problem hiding this comment.
Code Review
This pull request performs a major cleanup by removing outdated documentation, example scripts, and associated tests for features like low-precision training, on-policy distillation, multi-task evaluation, and several specific model configurations. It also introduces minor backend fixes, such as disabling temperature scaling for value outputs in Megatron loss calculations, and adds a new unit test test_loss.py. The review feedback correctly identifies a potential test flakiness issue in the new unit test where sys.modules.setdefault is used to register mock modules, and provides a solid suggestion to directly assign to sys.modules instead.
| sys.modules.setdefault("megatron", megatron_mod) | ||
| sys.modules.setdefault("megatron.core", megatron_core) |
There was a problem hiding this comment.
Using sys.modules.setdefault to register mock modules can lead to flaky or failing tests if another test in the suite has already imported megatron (since setdefault will not overwrite the existing entry in sys.modules). Directly assigning to sys.modules ensures that this unit test always uses the intended mock modules regardless of the test execution order.
| sys.modules.setdefault("megatron", megatron_mod) | |
| sys.modules.setdefault("megatron.core", megatron_core) | |
| sys.modules["megatron"] = megatron_mod | |
| sys.modules["megatron.core"] = megatron_core |
8cac252 to
b323b34
Compare
Port of THUDM/slime#1942 (update docs). Brought the technical doc updates (customization.md agentic-workflows section, trace.md, pd-disaggregation.md, en+zh), rebranded slime->vime / sglang_rollout->vllm_rollout / SGLang->vLLM. Divergence handling: - README.md / README_zh.md: kept vime's version (slime's changes were SGLang-branded architecture text + 'Projects built upon slime' marketing + an agentic-examples list — vime uses vLLM and trimmed examples in #126). - Dropped the broken examples/search-r1 link from the added customization text (trimmed in vime); kept examples/multi_agent + examples/fully_async (exist). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
…80/#1967/#1938/#1988) (#138) * [CI] add reward-fn cpu tests + cpu-on-main trigger (port slime #1939+#1940) Port of THUDM/slime#1939 (add more cpu ci) + #1940 (run cpu test on main): - tests/test_rm_{deepscaler,f1,gpqa,math,math_dapo}.py: pure-cpu unit tests for vime/rollout/rm_hub reward fns (107 tests, validated locally). - vime/utils/misc.py: lazy 'import ray' so cpu-only paths (rm_hub scoring, plugin contracts) don't require ray. - pr-test.yml.j2: register the 5 rm tests in the cpu (num_gpus:0) matrix; enable push-to-main trigger gated to cpu jobs only (GPU/self-hosted jobs stay PR/label-gated so push never burns the fleet). slime #1939 also added tests/test_sample.py, which references Sample.rollout_id (introduced by the #1926-chain / vime PR #119) — that test is stacked on #119 instead of here. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * [Doc] sync customization/trace/pd-disagg docs (port slime #1942) Port of THUDM/slime#1942 (update docs). Brought the technical doc updates (customization.md agentic-workflows section, trace.md, pd-disaggregation.md, en+zh), rebranded slime->vime / sglang_rollout->vllm_rollout / SGLang->vLLM. Divergence handling: - README.md / README_zh.md: kept vime's version (slime's changes were SGLang-branded architecture text + 'Projects built upon slime' marketing + an agentic-examples list — vime uses vLLM and trimmed examples in #126). - Dropped the broken examples/search-r1 link from the added customization text (trimmed in vime); kept examples/multi_agent + examples/fully_async (exist). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * [Chore] remove redundant replay tool + harden unwrap_model import (port slime #1943) Port of THUDM/slime#1943 (remove redundant file): - remove tools/replay_openai_jsonl.py (redundant). - vime/backends/megatron_utils/model.py: make unwrap_model import resilient to the Megatron move from megatron.core.utils to megatron.core.pipeline_parallel.utils (try/except fallback). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * [Sync][C] FLOPs MLA fix, PYTHONUNBUFFERED typo, spec-config guard, fault-tolerance docs Mechanical / docs sweep for the slime→vime sync (mega-PR C additions, on top of the cherry-picked #1939+#1940 / #1942 / #1943): - slime #1980: fix FLOPs accounting for non-MLA attention. Gate the MLA-shaped qkv/attention flop formulas on `multi_latent_attention` instead of inferring MLA from q_lora_rank/qk_pos_emb_head_dim/v_head_dim (which misattributed flops for non-MLA models). vime/utils/flops_utils.py. - slime #1967: fix PYTHONBUFFERED=16 typo -> PYTHONUNBUFFERED=1 across 12 scripts + command_utils.py (both name and value, matching slime canonical). - slime #1938: guard `vllm_speculative_config` read in _compute_spec_metrics with getattr(...) so --debug-train-only (skip_vllm leaves the attr unset) doesn't AttributeError. vime/ray/rollout.py. - slime #1988: rewrite docs/en/advanced/fault-tolerance.md (scope, health checks, debug/replay path, production pattern), rebranded sglang→vllm / slime→vime, /health_generate→/health, link vllm-config.md. Dropped from this PR after source review: - #1987/#1990 (R3 ci ref/logprob+kl guards) — already in main via vime #93 (b1009dd "Backports slime #1987 + #1990"). - #1974/#1989 — touch examples/coding_agent_rl + its README bullet; that feature is not in vime yet (lands with the agent mega-PR). - #1975 — sglang conda-ci image resolution + a test vime already has + build_conda micromamba fix; no real version-file port for vime. Deferred to after the rollout-data-model PR: #1986/#1985 (they edit that PR's tests). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs(mega-C): faithfully port slime #1942 docs (fault-tolerance/pd-disagg/trace) Audit (一模一样): mega-C's original #1942 doc port was incomplete — several docs were stubs or truncated vs slime. Port them faithfully (modulo sglang→vllm): - fault-tolerance.md en: add trailing (trainer-failure note + Related Docs). - fault-tolerance.md zh: was a 13-line stub → full 76-line port. - pd-disaggregation.md en+zh: were 7-line stubs → full 87-line port. - trace.md en+zh: port slime's build_meta example as build_vllm_meta_trace_attrs(output) (vime's real signature). Translation: SGLang→vLLM, /health_generate→/health (vime's real endpoint), sglang-config→vllm-config, `sglang:`→`vllm:` YAML key (vime vllm_config.py requires the 'vllm' key), server→engine; `--prefill-num-servers` flag preserved. All 6 files now match slime line counts (76/76, 87/87, 119/119). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(mega-C): port slime #1942 customization gaps (multi-sample return + teacher_log_probs) Surgically add the two slime #1942 additions that mega-C missed, keeping vime's legitimate divergences (vllm_rollout path, VIME_CONTRACT_* env, search-r1 removed in #74): - "#### Returning multiple training samples for one prompt" section (en+zh): custom_generate may return list[Sample] with shared group_id; signature updated to `-> Sample | list[Sample]`. - `teacher_log_probs` field in the Sample-fields dict (en+zh). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * [Sync][C] align to slime exactly: drop #1961 over-port, fix #1938/#1943 placement - customization.md (en+zh): revert custom_generate signature to `-> Sample` and remove the "Returning multiple training samples" section. That content is from slime #1961 (not #1942, despite the prior commit message) and documents Sample.group_id / Sample.rollout_id + list[Sample] return — all tied to the deferred rollout-data-model refactor that vime does not yet implement (verified: 0 occurrences of group_id/rollout_id in types.py and the rollout loop). Defers with #1986/#1985. - model.py: move the unwrap_model try/except to slime #1943's exact position (after the tqdm import) — import block now byte-identical to slime. - rollout.py: drop the 4-line comment around the spec-metrics getattr guard; slime #1938 is a pure one-line change with no added comment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * [Sync][C] cleanup sglang/dead-link residue: /health_generate help text + search-r1 refs - arguments.py: --rollout-health-check-{interval,timeout} help text said `/health_generate` (sglang's endpoint) but the code hits `/health` (vllm_engine.py). Fix the help strings to `/health`. - customization.md (en+zh) + index.rst (en+zh): remove the dead `examples/search-r1` references — that example does not exist in vime. Note: the method name `VLLMEngine.health_generate()` (vllm_engine.py) and its caller (health_monitor.py) keep the sglang-flavored name but already hit `/health`; renaming the method is a separate change, left out here. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * [Sync][C] remove dead examples/retool toctree refs examples/retool does not exist in vime (same dead-link situation as search-r1); drop it from the docs toctree in index.rst (en+zh). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * [Sync][C] docs: add Ray Distributed Debugger step-by-step section (port slime #2001) Appends the 'Step-by-Step Debugging with Ray Distributed Debugger' section (debugpy + RAY_DEBUG_POSTMORTEM + VS Code attach) to developer_guide/debug.md (en + zh). Content is engine-neutral (Ray/debugpy/VS Code), ported verbatim; no sglang->vllm translation needed. --------- Signed-off-by: aoshen02 <aoshen@inferact.ai> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Restore files that were either deleted by #126 ("trim examples to qwen3 only") or never synced from slime: **Reverted from pre-#126 (translated):** - scripts/low_precision/run-qwen3-4b-fp8.sh - scripts/low_precision/run-qwen3-30b-a3b-fp8.sh - scripts/run-glm4-9B.sh - scripts/run-moonlight-16B-A3B.sh - scripts/run-qwen3-4B-base-sft.sh - scripts/run-qwen3-32B.sh - scripts/run-qwen3.5-35B-A3B-sft.sh **New from slime@44d29ee (translated):** - docs/en/get_started/agent.md - examples/fully_async/run-qwen2.5-0.5B-fully_async.sh All sglang engine flags translated to vllm equivalents (§2.4). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Restore files that were either deleted by #126 ("trim examples to qwen3 only") or never synced from slime: **Reverted from pre-#126 (translated):** - scripts/low_precision/run-qwen3-4b-fp8.sh - scripts/low_precision/run-qwen3-30b-a3b-fp8.sh - scripts/run-glm4-9B.sh - scripts/run-moonlight-16B-A3B.sh - scripts/run-qwen3-4B-base-sft.sh - scripts/run-qwen3-32B.sh - scripts/run-qwen3.5-35B-A3B-sft.sh **New from slime@44d29ee (translated):** - docs/en/get_started/agent.md - examples/fully_async/run-qwen2.5-0.5B-fully_async.sh All sglang engine flags translated to vllm equivalents (§2.4). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Restore files that were either deleted by #126 ("trim examples to qwen3 only") or never synced from slime: **Reverted from pre-#126 (translated):** - scripts/low_precision/run-qwen3-4b-fp8.sh - scripts/low_precision/run-qwen3-30b-a3b-fp8.sh - scripts/run-glm4-9B.sh - scripts/run-moonlight-16B-A3B.sh - scripts/run-qwen3-4B-base-sft.sh - scripts/run-qwen3-32B.sh - scripts/run-qwen3.5-35B-A3B-sft.sh **New from slime@44d29ee (translated):** - docs/en/get_started/agent.md - examples/fully_async/run-qwen2.5-0.5B-fully_async.sh All sglang engine flags translated to vllm equivalents (§2.4). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Restore files that were either deleted by #126 ("trim examples to qwen3 only") or never synced from slime: **Reverted from pre-#126 (translated):** - scripts/low_precision/run-qwen3-4b-fp8.sh - scripts/low_precision/run-qwen3-30b-a3b-fp8.sh - scripts/run-glm4-9B.sh - scripts/run-moonlight-16B-A3B.sh - scripts/run-qwen3-4B-base-sft.sh - scripts/run-qwen3-32B.sh - scripts/run-qwen3.5-35B-A3B-sft.sh **New from slime@44d29ee (translated):** - docs/en/get_started/agent.md - examples/fully_async/run-qwen2.5-0.5B-fully_async.sh All sglang engine flags translated to vllm equivalents (§2.4). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Restore files that were either deleted by #126 ("trim examples to qwen3 only") or never synced from slime: **Reverted from pre-#126 (translated):** - scripts/low_precision/run-qwen3-4b-fp8.sh - scripts/low_precision/run-qwen3-30b-a3b-fp8.sh - scripts/run-glm4-9B.sh - scripts/run-moonlight-16B-A3B.sh - scripts/run-qwen3-4B-base-sft.sh - scripts/run-qwen3-32B.sh - scripts/run-qwen3.5-35B-A3B-sft.sh **New from slime@44d29ee (translated):** - docs/en/get_started/agent.md - examples/fully_async/run-qwen2.5-0.5B-fully_async.sh All sglang engine flags translated to vllm equivalents (§2.4). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Restore files that were either deleted by #126 ("trim examples to qwen3 only") or never synced from slime: **Reverted from pre-#126 (translated):** - scripts/low_precision/run-qwen3-4b-fp8.sh - scripts/low_precision/run-qwen3-30b-a3b-fp8.sh - scripts/run-glm4-9B.sh - scripts/run-moonlight-16B-A3B.sh - scripts/run-qwen3-4B-base-sft.sh - scripts/run-qwen3-32B.sh - scripts/run-qwen3.5-35B-A3B-sft.sh **New from slime@44d29ee (translated):** - docs/en/get_started/agent.md - examples/fully_async/run-qwen2.5-0.5B-fully_async.sh All sglang engine flags translated to vllm equivalents (§2.4). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… 20B support (#260) * restore: bring back deleted examples, scripts, and agent doc Restore files that were either deleted by #126 ("trim examples to qwen3 only") or never synced from slime: **Reverted from pre-#126 (translated):** - scripts/low_precision/run-qwen3-4b-fp8.sh - scripts/low_precision/run-qwen3-30b-a3b-fp8.sh - scripts/run-glm4-9B.sh - scripts/run-moonlight-16B-A3B.sh - scripts/run-qwen3-4B-base-sft.sh - scripts/run-qwen3-32B.sh - scripts/run-qwen3.5-35B-A3B-sft.sh **New from slime@44d29ee (translated):** - docs/en/get_started/agent.md - examples/fully_async/run-qwen2.5-0.5B-fully_async.sh All sglang engine flags translated to vllm equivalents (§2.4). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: unify pkill pattern to '[v]llm serve|VLL[M]::' Standardize all scripts to use the bracket-escaped pkill pattern that avoids matching pkill itself and also catches vLLM's renamed subprocesses (VLLM::EngineCore, VLLM::Worker_TP*). Matches the canonical pattern in command_utils.py. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * scripts: complete slime-exact translation of all 29 run scripts Translate all slime scripts to vime following SGLANG_TO_VLLM_TRANSLATION.md: - sglang→vllm prefix swap for CLI flags and variables - _slime→_vime for checkpoint paths - EP: --sglang-ep-size N → --vllm-enable-expert-parallel (boolean) - Speculative: multi-param → --vllm-speculative-config JSON (§5.2) - Delete genuinely sglang-coupled params (DP-attention, DeepEP, NSA, etc.) - flashinfer → FLASHINFER case fix (§2.4) 23 new scripts + 6 existing updated to match slime@cutoff. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(scripts): correct model config source path in FP8 low_precision scripts The FP8 scripts used `${SCRIPT_DIR}/../scripts/models/` which resolves to `scripts/scripts/models/` (non-existent). Changed to `../models/` to match the INT4 scripts. Same fix as slime PR #2094. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(gpt-oss): fused BF16 format, bridge API patch, bshd qkv format Three fixes needed to run GPT-OSS 20B RLHF on vLLM backend: 1. hf_weight_iterator_bridge: match Megatron-Bridge 0.5.0 API _patch_bridge_expert_cache_to_cpu monkey-patches GPTOSSBridge. maybe_modify_converted_hf_weight gained a 4th `hf_state_dict` parameter; the patched wrapper only accepted 3, causing TypeError during weight sync. 2. run-gpt-oss-20B: point --hf-checkpoint at fused BF16 format vLLM's _load_weights_other expects gate_up_proj [E, hidden, 2*ffn] (fused). The old per-expert split format (experts.{e}.gate_proj.weight) causes KeyError on bias loading. Use tools/convert_gpt_oss_to_fused.py to convert an existing per-expert checkpoint, or re-run preprocess_gpt_oss.py to produce fused format directly. 3. run-gpt-oss-20B: add --qkv-format bshd + fix seq-length GPT-OSS uses learnable softmax (sink attention). TransformerEngine disables all attention backends when softmax_type=learnable and qkv_format=thd (packed sequences). --qkv-format bshd avoids this. --use-dynamic-batch-size is incompatible with bshd; replaced with fixed --seq-length 10240 (covers 8192 max response + prompt headroom). tools/convert_gpt_oss_to_fused.py: new tool to convert per-expert BF16 checkpoint (output of old preprocess_gpt_oss.py) to the fused HF format expected by vLLM without re-running the slow MXFP4 dequantization. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(scripts): replace pkill -9 vllm with precise -f pattern (21 files) pkill -9 vllm matches any process named "vllm" and can inadvertently kill unrelated vllm processes (e.g. background services). Use the same pattern as PR #220 which targets only vllm serve and Ray VLL[M]:: actors: pkill -9 -f '[v]llm serve|VLL[M]::' Also updates the inline form used in multi-node SSH worker restart commands (run-qwen3-235B-A22B*.sh, run-qwen3.5-27B.sh, etc.). Skipped: scripts/run-gpt-oss-20B.sh (uses pkill -9 -f "vllm serve" already), scripts/run-minimax-m2.sh and run-glm4.7-*.sh (already used -f "vllm serve"). Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * chore(scripts): remove run-qwen3-4B-amd.sh from this PR AMD-specific script is out of scope for the gb300-complete-port PR. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * sync(docs+scripts): port docs/examples from slime-44d29ee, fix script translations - Add missing EN/ZH docs: low-precision, on-policy-distillation, get_started/agent, pd-disaggregation (heterogeneous server groups fix), examples zh docs - Add missing examples: on_policy_distillation, eval_multi_task, delta_weight_sync, geo3k images - Fix vLLM flag translations across all example docs: - --vllm-mem-fraction-static → --vllm-gpu-memory-utilization - Remove non-existent dp-attention flags (--vllm-enable-dp-attention, --vllm-dp-size, --vllm-moe-dense-tp-size, --vllm-enable-dp-lm-head, --vllm-ep-size) - --vllm-ep-num-redundant-experts → --vllm-eplb-config - --vllm-cuda-graph-bs → --vllm-max-cudagraph-capture-size - sglang speculative flags → --vllm-speculative-config JSON - GLM-4.7 MTP: method=eagle → method=mtp, num_speculative_tokens=4 → 3 - sgl-router → vllm-router; THUDM/vime → vllm-project/vime - Fix scripts: restore run-kimi-k2-Instruct/Thinking/qwen3-4B/qwen3-235B-A22B to slime-44d29ee-as-vime + pkill precision fix only; restore int4 python3 path Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * revert(scripts): pkill -9 -f pattern back to pkill -9 vllm, align with slime Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * revert(pkill): align all remaining kill patterns with slime (pkill -9 vllm) Covers examples/, docs/, tests/, and vime/utils -- previously missed in the scripts/ revert. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * chore: remove gpt-oss-20B script and convert tool (moved to separate PR) Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… 20B support (vllm-project#260) * restore: bring back deleted examples, scripts, and agent doc Restore files that were either deleted by vllm-project#126 ("trim examples to qwen3 only") or never synced from slime: **Reverted from pre-vllm-project#126 (translated):** - scripts/low_precision/run-qwen3-4b-fp8.sh - scripts/low_precision/run-qwen3-30b-a3b-fp8.sh - scripts/run-glm4-9B.sh - scripts/run-moonlight-16B-A3B.sh - scripts/run-qwen3-4B-base-sft.sh - scripts/run-qwen3-32B.sh - scripts/run-qwen3.5-35B-A3B-sft.sh **New from slime@44d29ee (translated):** - docs/en/get_started/agent.md - examples/fully_async/run-qwen2.5-0.5B-fully_async.sh All sglang engine flags translated to vllm equivalents (§2.4). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: unify pkill pattern to '[v]llm serve|VLL[M]::' Standardize all scripts to use the bracket-escaped pkill pattern that avoids matching pkill itself and also catches vLLM's renamed subprocesses (VLLM::EngineCore, VLLM::Worker_TP*). Matches the canonical pattern in command_utils.py. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * scripts: complete slime-exact translation of all 29 run scripts Translate all slime scripts to vime following SGLANG_TO_VLLM_TRANSLATION.md: - sglang→vllm prefix swap for CLI flags and variables - _slime→_vime for checkpoint paths - EP: --sglang-ep-size N → --vllm-enable-expert-parallel (boolean) - Speculative: multi-param → --vllm-speculative-config JSON (§5.2) - Delete genuinely sglang-coupled params (DP-attention, DeepEP, NSA, etc.) - flashinfer → FLASHINFER case fix (§2.4) 23 new scripts + 6 existing updated to match slime@cutoff. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(scripts): correct model config source path in FP8 low_precision scripts The FP8 scripts used `${SCRIPT_DIR}/../scripts/models/` which resolves to `scripts/scripts/models/` (non-existent). Changed to `../models/` to match the INT4 scripts. Same fix as slime PR #2094. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(gpt-oss): fused BF16 format, bridge API patch, bshd qkv format Three fixes needed to run GPT-OSS 20B RLHF on vLLM backend: 1. hf_weight_iterator_bridge: match Megatron-Bridge 0.5.0 API _patch_bridge_expert_cache_to_cpu monkey-patches GPTOSSBridge. maybe_modify_converted_hf_weight gained a 4th `hf_state_dict` parameter; the patched wrapper only accepted 3, causing TypeError during weight sync. 2. run-gpt-oss-20B: point --hf-checkpoint at fused BF16 format vLLM's _load_weights_other expects gate_up_proj [E, hidden, 2*ffn] (fused). The old per-expert split format (experts.{e}.gate_proj.weight) causes KeyError on bias loading. Use tools/convert_gpt_oss_to_fused.py to convert an existing per-expert checkpoint, or re-run preprocess_gpt_oss.py to produce fused format directly. 3. run-gpt-oss-20B: add --qkv-format bshd + fix seq-length GPT-OSS uses learnable softmax (sink attention). TransformerEngine disables all attention backends when softmax_type=learnable and qkv_format=thd (packed sequences). --qkv-format bshd avoids this. --use-dynamic-batch-size is incompatible with bshd; replaced with fixed --seq-length 10240 (covers 8192 max response + prompt headroom). tools/convert_gpt_oss_to_fused.py: new tool to convert per-expert BF16 checkpoint (output of old preprocess_gpt_oss.py) to the fused HF format expected by vLLM without re-running the slow MXFP4 dequantization. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(scripts): replace pkill -9 vllm with precise -f pattern (21 files) pkill -9 vllm matches any process named "vllm" and can inadvertently kill unrelated vllm processes (e.g. background services). Use the same pattern as PR vllm-project#220 which targets only vllm serve and Ray VLL[M]:: actors: pkill -9 -f '[v]llm serve|VLL[M]::' Also updates the inline form used in multi-node SSH worker restart commands (run-qwen3-235B-A22B*.sh, run-qwen3.5-27B.sh, etc.). Skipped: scripts/run-gpt-oss-20B.sh (uses pkill -9 -f "vllm serve" already), scripts/run-minimax-m2.sh and run-glm4.7-*.sh (already used -f "vllm serve"). Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * chore(scripts): remove run-qwen3-4B-amd.sh from this PR AMD-specific script is out of scope for the gb300-complete-port PR. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * sync(docs+scripts): port docs/examples from slime-44d29ee, fix script translations - Add missing EN/ZH docs: low-precision, on-policy-distillation, get_started/agent, pd-disaggregation (heterogeneous server groups fix), examples zh docs - Add missing examples: on_policy_distillation, eval_multi_task, delta_weight_sync, geo3k images - Fix vLLM flag translations across all example docs: - --vllm-mem-fraction-static → --vllm-gpu-memory-utilization - Remove non-existent dp-attention flags (--vllm-enable-dp-attention, --vllm-dp-size, --vllm-moe-dense-tp-size, --vllm-enable-dp-lm-head, --vllm-ep-size) - --vllm-ep-num-redundant-experts → --vllm-eplb-config - --vllm-cuda-graph-bs → --vllm-max-cudagraph-capture-size - sglang speculative flags → --vllm-speculative-config JSON - GLM-4.7 MTP: method=eagle → method=mtp, num_speculative_tokens=4 → 3 - sgl-router → vllm-router; THUDM/vime → vllm-project/vime - Fix scripts: restore run-kimi-k2-Instruct/Thinking/qwen3-4B/qwen3-235B-A22B to slime-44d29ee-as-vime + pkill precision fix only; restore int4 python3 path Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * revert(scripts): pkill -9 -f pattern back to pkill -9 vllm, align with slime Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * revert(pkill): align all remaining kill patterns with slime (pkill -9 vllm) Covers examples/, docs/, tests/, and vime/utils -- previously missed in the scripts/ revert. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * chore: remove gpt-oss-20B script and convert tool (moved to separate PR) Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
This PR removes the public example set down to Qwen3-4B and Qwen3-30B-A3B, plus the related OPD and low-precision example/docs entries we already agreed to drop.
It is intentionally separate from the slime->vime rename PR so the rename can be rebased cleanly after this lands.