Skip to content

chore: trim examples to qwen3 only - #126

Merged
CalvinXKY merged 1 commit into
mainfrom
chore/keep-qwen-examples
Jun 2, 2026
Merged

chore: trim examples to qwen3 only#126
CalvinXKY merged 1 commit into
mainfrom
chore/keep-qwen-examples

Conversation

@aoshen02

@aoshen02 aoshen02 commented Jun 2, 2026

Copy link
Copy Markdown
Collaborator

This PR removes the public example set down to Qwen3-4B and Qwen3-30B-A3B, plus the related OPD and low-precision example/docs entries we already agreed to drop.

It is intentionally separate from the slime->vime rename PR so the rename can be rebased cleanly after this lands.

@aoshen02
aoshen02 force-pushed the chore/keep-qwen-examples branch from caf0638 to 684be18 Compare June 2, 2026 11:54

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request performs a major cleanup by removing outdated documentation, example scripts, and associated tests for features like low-precision training, on-policy distillation, multi-task evaluation, and several specific model configurations. It also introduces minor backend fixes, such as disabling temperature scaling for value outputs in Megatron loss calculations, and adds a new unit test test_loss.py. The review feedback correctly identifies a potential test flakiness issue in the new unit test where sys.modules.setdefault is used to register mock modules, and provides a solid suggestion to directly assign to sys.modules instead.

Comment on lines +18 to +19
sys.modules.setdefault("megatron", megatron_mod)
sys.modules.setdefault("megatron.core", megatron_core)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using sys.modules.setdefault to register mock modules can lead to flaky or failing tests if another test in the suite has already imported megatron (since setdefault will not overwrite the existing entry in sys.modules). Directly assigning to sys.modules ensures that this unit test always uses the intended mock modules regardless of the test execution order.

Suggested change
sys.modules.setdefault("megatron", megatron_mod)
sys.modules.setdefault("megatron.core", megatron_core)
sys.modules["megatron"] = megatron_mod
sys.modules["megatron.core"] = megatron_core

@CalvinXKY CalvinXKY left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGYM. Ensure it doesn't affect the CI.

@aoshen02
aoshen02 force-pushed the chore/keep-qwen-examples branch from 8cac252 to b323b34 Compare June 2, 2026 12:39
@CalvinXKY
CalvinXKY merged commit 9ecf10f into main Jun 2, 2026
10 of 13 checks passed
aoshen02 added a commit that referenced this pull request Jun 5, 2026
Port of THUDM/slime#1942 (update docs). Brought the technical doc updates
(customization.md agentic-workflows section, trace.md, pd-disaggregation.md,
en+zh), rebranded slime->vime / sglang_rollout->vllm_rollout / SGLang->vLLM.

Divergence handling:
- README.md / README_zh.md: kept vime's version (slime's changes were
  SGLang-branded architecture text + 'Projects built upon slime' marketing +
  an agentic-examples list — vime uses vLLM and trimmed examples in #126).
- Dropped the broken examples/search-r1 link from the added customization text
  (trimmed in vime); kept examples/multi_agent + examples/fully_async (exist).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
CalvinXKY pushed a commit that referenced this pull request Jun 7, 2026
…80/#1967/#1938/#1988) (#138)

* [CI] add reward-fn cpu tests + cpu-on-main trigger (port slime #1939+#1940)

Port of THUDM/slime#1939 (add more cpu ci) + #1940 (run cpu test on main):
- tests/test_rm_{deepscaler,f1,gpqa,math,math_dapo}.py: pure-cpu unit tests
  for vime/rollout/rm_hub reward fns (107 tests, validated locally).
- vime/utils/misc.py: lazy 'import ray' so cpu-only paths (rm_hub scoring,
  plugin contracts) don't require ray.
- pr-test.yml.j2: register the 5 rm tests in the cpu (num_gpus:0) matrix;
  enable push-to-main trigger gated to cpu jobs only (GPU/self-hosted jobs
  stay PR/label-gated so push never burns the fleet).

slime #1939 also added tests/test_sample.py, which references Sample.rollout_id
(introduced by the #1926-chain / vime PR #119) — that test is stacked on #119
instead of here.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>

* [Doc] sync customization/trace/pd-disagg docs (port slime #1942)

Port of THUDM/slime#1942 (update docs). Brought the technical doc updates
(customization.md agentic-workflows section, trace.md, pd-disaggregation.md,
en+zh), rebranded slime->vime / sglang_rollout->vllm_rollout / SGLang->vLLM.

Divergence handling:
- README.md / README_zh.md: kept vime's version (slime's changes were
  SGLang-branded architecture text + 'Projects built upon slime' marketing +
  an agentic-examples list — vime uses vLLM and trimmed examples in #126).
- Dropped the broken examples/search-r1 link from the added customization text
  (trimmed in vime); kept examples/multi_agent + examples/fully_async (exist).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>

* [Chore] remove redundant replay tool + harden unwrap_model import (port slime #1943)

Port of THUDM/slime#1943 (remove redundant file):
- remove tools/replay_openai_jsonl.py (redundant).
- vime/backends/megatron_utils/model.py: make unwrap_model import resilient
  to the Megatron move from megatron.core.utils to
  megatron.core.pipeline_parallel.utils (try/except fallback).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>

* [Sync][C] FLOPs MLA fix, PYTHONUNBUFFERED typo, spec-config guard, fault-tolerance docs

Mechanical / docs sweep for the slime→vime sync (mega-PR C additions, on top of
the cherry-picked #1939+#1940 / #1942 / #1943):

- slime #1980: fix FLOPs accounting for non-MLA attention. Gate the MLA-shaped
  qkv/attention flop formulas on `multi_latent_attention` instead of inferring
  MLA from q_lora_rank/qk_pos_emb_head_dim/v_head_dim (which misattributed flops
  for non-MLA models). vime/utils/flops_utils.py.
- slime #1967: fix PYTHONBUFFERED=16 typo -> PYTHONUNBUFFERED=1 across 12 scripts
  + command_utils.py (both name and value, matching slime canonical).
- slime #1938: guard `vllm_speculative_config` read in _compute_spec_metrics with
  getattr(...) so --debug-train-only (skip_vllm leaves the attr unset) doesn't
  AttributeError. vime/ray/rollout.py.
- slime #1988: rewrite docs/en/advanced/fault-tolerance.md (scope, health checks,
  debug/replay path, production pattern), rebranded sglang→vllm / slime→vime,
  /health_generate→/health, link vllm-config.md.

Dropped from this PR after source review:
- #1987/#1990 (R3 ci ref/logprob+kl guards) — already in main via vime #93
  (b1009dd "Backports slime #1987 + #1990").
- #1974/#1989 — touch examples/coding_agent_rl + its README bullet; that feature
  is not in vime yet (lands with the agent mega-PR).
- #1975 — sglang conda-ci image resolution + a test vime already has + build_conda
  micromamba fix; no real version-file port for vime.
Deferred to after the rollout-data-model PR: #1986/#1985 (they edit that PR's tests).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>

* docs(mega-C): faithfully port slime #1942 docs (fault-tolerance/pd-disagg/trace)

Audit (一模一样): mega-C's original #1942 doc port was incomplete — several docs
were stubs or truncated vs slime. Port them faithfully (modulo sglang→vllm):

- fault-tolerance.md en: add trailing (trainer-failure note + Related Docs).
- fault-tolerance.md zh: was a 13-line stub → full 76-line port.
- pd-disaggregation.md en+zh: were 7-line stubs → full 87-line port.
- trace.md en+zh: port slime's build_meta example as
  build_vllm_meta_trace_attrs(output) (vime's real signature).

Translation: SGLang→vLLM, /health_generate→/health (vime's real endpoint),
sglang-config→vllm-config, `sglang:`→`vllm:` YAML key (vime vllm_config.py
requires the 'vllm' key), server→engine; `--prefill-num-servers` flag preserved.
All 6 files now match slime line counts (76/76, 87/87, 119/119).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(mega-C): port slime #1942 customization gaps (multi-sample return + teacher_log_probs)

Surgically add the two slime #1942 additions that mega-C missed, keeping vime's
legitimate divergences (vllm_rollout path, VIME_CONTRACT_* env, search-r1 removed
in #74):
- "#### Returning multiple training samples for one prompt" section (en+zh):
  custom_generate may return list[Sample] with shared group_id; signature
  updated to `-> Sample | list[Sample]`.
- `teacher_log_probs` field in the Sample-fields dict (en+zh).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* [Sync][C] align to slime exactly: drop #1961 over-port, fix #1938/#1943 placement

- customization.md (en+zh): revert custom_generate signature to `-> Sample`
  and remove the "Returning multiple training samples" section. That content
  is from slime #1961 (not #1942, despite the prior commit message) and
  documents Sample.group_id / Sample.rollout_id + list[Sample] return — all
  tied to the deferred rollout-data-model refactor that vime does not yet
  implement (verified: 0 occurrences of group_id/rollout_id in types.py and
  the rollout loop). Defers with #1986/#1985.
- model.py: move the unwrap_model try/except to slime #1943's exact position
  (after the tqdm import) — import block now byte-identical to slime.
- rollout.py: drop the 4-line comment around the spec-metrics getattr guard;
  slime #1938 is a pure one-line change with no added comment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* [Sync][C] cleanup sglang/dead-link residue: /health_generate help text + search-r1 refs

- arguments.py: --rollout-health-check-{interval,timeout} help text said
  `/health_generate` (sglang's endpoint) but the code hits `/health`
  (vllm_engine.py). Fix the help strings to `/health`.
- customization.md (en+zh) + index.rst (en+zh): remove the dead
  `examples/search-r1` references — that example does not exist in vime.

Note: the method name `VLLMEngine.health_generate()` (vllm_engine.py) and its
caller (health_monitor.py) keep the sglang-flavored name but already hit
`/health`; renaming the method is a separate change, left out here.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* [Sync][C] remove dead examples/retool toctree refs

examples/retool does not exist in vime (same dead-link situation as
search-r1); drop it from the docs toctree in index.rst (en+zh).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* [Sync][C] docs: add Ray Distributed Debugger step-by-step section (port slime #2001)

Appends the 'Step-by-Step Debugging with Ray Distributed Debugger' section
(debugpy + RAY_DEBUG_POSTMORTEM + VS Code attach) to developer_guide/debug.md
(en + zh). Content is engine-neutral (Ray/debugpy/VS Code), ported verbatim;
no sglang->vllm translation needed.

---------

Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@aoshen02
aoshen02 deleted the chore/keep-qwen-examples branch June 8, 2026 14:17
aoshen02 added a commit that referenced this pull request Jun 9, 2026
Restore files that were either deleted by #126 ("trim examples to
qwen3 only") or never synced from slime:

**Reverted from pre-#126 (translated):**
- scripts/low_precision/run-qwen3-4b-fp8.sh
- scripts/low_precision/run-qwen3-30b-a3b-fp8.sh
- scripts/run-glm4-9B.sh
- scripts/run-moonlight-16B-A3B.sh
- scripts/run-qwen3-4B-base-sft.sh
- scripts/run-qwen3-32B.sh
- scripts/run-qwen3.5-35B-A3B-sft.sh

**New from slime@44d29ee (translated):**
- docs/en/get_started/agent.md
- examples/fully_async/run-qwen2.5-0.5B-fully_async.sh

All sglang engine flags translated to vllm equivalents (§2.4).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
aoshen02 added a commit that referenced this pull request Jun 9, 2026
Restore files that were either deleted by #126 ("trim examples to
qwen3 only") or never synced from slime:

**Reverted from pre-#126 (translated):**
- scripts/low_precision/run-qwen3-4b-fp8.sh
- scripts/low_precision/run-qwen3-30b-a3b-fp8.sh
- scripts/run-glm4-9B.sh
- scripts/run-moonlight-16B-A3B.sh
- scripts/run-qwen3-4B-base-sft.sh
- scripts/run-qwen3-32B.sh
- scripts/run-qwen3.5-35B-A3B-sft.sh

**New from slime@44d29ee (translated):**
- docs/en/get_started/agent.md
- examples/fully_async/run-qwen2.5-0.5B-fully_async.sh

All sglang engine flags translated to vllm equivalents (§2.4).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
aoshen02 added a commit that referenced this pull request Jun 9, 2026
Restore files that were either deleted by #126 ("trim examples to
qwen3 only") or never synced from slime:

**Reverted from pre-#126 (translated):**
- scripts/low_precision/run-qwen3-4b-fp8.sh
- scripts/low_precision/run-qwen3-30b-a3b-fp8.sh
- scripts/run-glm4-9B.sh
- scripts/run-moonlight-16B-A3B.sh
- scripts/run-qwen3-4B-base-sft.sh
- scripts/run-qwen3-32B.sh
- scripts/run-qwen3.5-35B-A3B-sft.sh

**New from slime@44d29ee (translated):**
- docs/en/get_started/agent.md
- examples/fully_async/run-qwen2.5-0.5B-fully_async.sh

All sglang engine flags translated to vllm equivalents (§2.4).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
aoshen02 added a commit that referenced this pull request Jun 9, 2026
Restore files that were either deleted by #126 ("trim examples to
qwen3 only") or never synced from slime:

**Reverted from pre-#126 (translated):**
- scripts/low_precision/run-qwen3-4b-fp8.sh
- scripts/low_precision/run-qwen3-30b-a3b-fp8.sh
- scripts/run-glm4-9B.sh
- scripts/run-moonlight-16B-A3B.sh
- scripts/run-qwen3-4B-base-sft.sh
- scripts/run-qwen3-32B.sh
- scripts/run-qwen3.5-35B-A3B-sft.sh

**New from slime@44d29ee (translated):**
- docs/en/get_started/agent.md
- examples/fully_async/run-qwen2.5-0.5B-fully_async.sh

All sglang engine flags translated to vllm equivalents (§2.4).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
aoshen02 added a commit that referenced this pull request Jun 9, 2026
Restore files that were either deleted by #126 ("trim examples to
qwen3 only") or never synced from slime:

**Reverted from pre-#126 (translated):**
- scripts/low_precision/run-qwen3-4b-fp8.sh
- scripts/low_precision/run-qwen3-30b-a3b-fp8.sh
- scripts/run-glm4-9B.sh
- scripts/run-moonlight-16B-A3B.sh
- scripts/run-qwen3-4B-base-sft.sh
- scripts/run-qwen3-32B.sh
- scripts/run-qwen3.5-35B-A3B-sft.sh

**New from slime@44d29ee (translated):**
- docs/en/get_started/agent.md
- examples/fully_async/run-qwen2.5-0.5B-fully_async.sh

All sglang engine flags translated to vllm equivalents (§2.4).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
aoshen02 added a commit that referenced this pull request Jun 9, 2026
Restore files that were either deleted by #126 ("trim examples to
qwen3 only") or never synced from slime:

**Reverted from pre-#126 (translated):**
- scripts/low_precision/run-qwen3-4b-fp8.sh
- scripts/low_precision/run-qwen3-30b-a3b-fp8.sh
- scripts/run-glm4-9B.sh
- scripts/run-moonlight-16B-A3B.sh
- scripts/run-qwen3-4B-base-sft.sh
- scripts/run-qwen3-32B.sh
- scripts/run-qwen3.5-35B-A3B-sft.sh

**New from slime@44d29ee (translated):**
- docs/en/get_started/agent.md
- examples/fully_async/run-qwen2.5-0.5B-fully_async.sh

All sglang engine flags translated to vllm equivalents (§2.4).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@aoshen02 aoshen02 mentioned this pull request Jun 21, 2026
14 tasks
CalvinXKY pushed a commit that referenced this pull request Jun 25, 2026
… 20B support (#260)

* restore: bring back deleted examples, scripts, and agent doc

Restore files that were either deleted by #126 ("trim examples to
qwen3 only") or never synced from slime:

**Reverted from pre-#126 (translated):**
- scripts/low_precision/run-qwen3-4b-fp8.sh
- scripts/low_precision/run-qwen3-30b-a3b-fp8.sh
- scripts/run-glm4-9B.sh
- scripts/run-moonlight-16B-A3B.sh
- scripts/run-qwen3-4B-base-sft.sh
- scripts/run-qwen3-32B.sh
- scripts/run-qwen3.5-35B-A3B-sft.sh

**New from slime@44d29ee (translated):**
- docs/en/get_started/agent.md
- examples/fully_async/run-qwen2.5-0.5B-fully_async.sh

All sglang engine flags translated to vllm equivalents (§2.4).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: unify pkill pattern to '[v]llm serve|VLL[M]::'

Standardize all scripts to use the bracket-escaped pkill pattern that
avoids matching pkill itself and also catches vLLM's renamed
subprocesses (VLLM::EngineCore, VLLM::Worker_TP*). Matches the
canonical pattern in command_utils.py.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* scripts: complete slime-exact translation of all 29 run scripts

Translate all slime scripts to vime following SGLANG_TO_VLLM_TRANSLATION.md:
- sglang→vllm prefix swap for CLI flags and variables
- _slime→_vime for checkpoint paths
- EP: --sglang-ep-size N → --vllm-enable-expert-parallel (boolean)
- Speculative: multi-param → --vllm-speculative-config JSON (§5.2)
- Delete genuinely sglang-coupled params (DP-attention, DeepEP, NSA, etc.)
- flashinfer → FLASHINFER case fix (§2.4)

23 new scripts + 6 existing updated to match slime@cutoff.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(scripts): correct model config source path in FP8 low_precision scripts

The FP8 scripts used `${SCRIPT_DIR}/../scripts/models/` which resolves
to `scripts/scripts/models/` (non-existent). Changed to `../models/`
to match the INT4 scripts. Same fix as slime PR #2094.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(gpt-oss): fused BF16 format, bridge API patch, bshd qkv format

Three fixes needed to run GPT-OSS 20B RLHF on vLLM backend:

1. hf_weight_iterator_bridge: match Megatron-Bridge 0.5.0 API
   _patch_bridge_expert_cache_to_cpu monkey-patches GPTOSSBridge.
   maybe_modify_converted_hf_weight gained a 4th `hf_state_dict`
   parameter; the patched wrapper only accepted 3, causing TypeError
   during weight sync.

2. run-gpt-oss-20B: point --hf-checkpoint at fused BF16 format
   vLLM's _load_weights_other expects gate_up_proj [E, hidden, 2*ffn]
   (fused). The old per-expert split format (experts.{e}.gate_proj.weight)
   causes KeyError on bias loading. Use tools/convert_gpt_oss_to_fused.py
   to convert an existing per-expert checkpoint, or re-run
   preprocess_gpt_oss.py to produce fused format directly.

3. run-gpt-oss-20B: add --qkv-format bshd + fix seq-length
   GPT-OSS uses learnable softmax (sink attention). TransformerEngine
   disables all attention backends when softmax_type=learnable and
   qkv_format=thd (packed sequences). --qkv-format bshd avoids this.
   --use-dynamic-batch-size is incompatible with bshd; replaced with
   fixed --seq-length 10240 (covers 8192 max response + prompt headroom).

tools/convert_gpt_oss_to_fused.py: new tool to convert per-expert BF16
checkpoint (output of old preprocess_gpt_oss.py) to the fused HF format
expected by vLLM without re-running the slow MXFP4 dequantization.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix(scripts): replace pkill -9 vllm with precise -f pattern (21 files)

pkill -9 vllm matches any process named "vllm" and can inadvertently
kill unrelated vllm processes (e.g. background services). Use the same
pattern as PR #220 which targets only vllm serve and Ray VLL[M]:: actors:

  pkill -9 -f '[v]llm serve|VLL[M]::'

Also updates the inline form used in multi-node SSH worker restart
commands (run-qwen3-235B-A22B*.sh, run-qwen3.5-27B.sh, etc.).

Skipped: scripts/run-gpt-oss-20B.sh (uses pkill -9 -f "vllm serve" already),
scripts/run-minimax-m2.sh and run-glm4.7-*.sh (already used -f "vllm serve").

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* chore(scripts): remove run-qwen3-4B-amd.sh from this PR

AMD-specific script is out of scope for the gb300-complete-port PR.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* sync(docs+scripts): port docs/examples from slime-44d29ee, fix script translations

- Add missing EN/ZH docs: low-precision, on-policy-distillation, get_started/agent,
  pd-disaggregation (heterogeneous server groups fix), examples zh docs
- Add missing examples: on_policy_distillation, eval_multi_task, delta_weight_sync,
  geo3k images
- Fix vLLM flag translations across all example docs:
  - --vllm-mem-fraction-static → --vllm-gpu-memory-utilization
  - Remove non-existent dp-attention flags (--vllm-enable-dp-attention, --vllm-dp-size,
    --vllm-moe-dense-tp-size, --vllm-enable-dp-lm-head, --vllm-ep-size)
  - --vllm-ep-num-redundant-experts → --vllm-eplb-config
  - --vllm-cuda-graph-bs → --vllm-max-cudagraph-capture-size
  - sglang speculative flags → --vllm-speculative-config JSON
  - GLM-4.7 MTP: method=eagle → method=mtp, num_speculative_tokens=4 → 3
  - sgl-router → vllm-router; THUDM/vime → vllm-project/vime
- Fix scripts: restore run-kimi-k2-Instruct/Thinking/qwen3-4B/qwen3-235B-A22B to
  slime-44d29ee-as-vime + pkill precision fix only; restore int4 python3 path

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* revert(scripts): pkill -9 -f pattern back to pkill -9 vllm, align with slime

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* revert(pkill): align all remaining kill patterns with slime (pkill -9 vllm)

Covers examples/, docs/, tests/, and vime/utils -- previously missed in
the scripts/ revert.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* chore: remove gpt-oss-20B script and convert tool (moved to separate PR)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
aoshen02 added a commit to aoshen02/vime that referenced this pull request Jul 15, 2026
… 20B support (vllm-project#260)

* restore: bring back deleted examples, scripts, and agent doc

Restore files that were either deleted by vllm-project#126 ("trim examples to
qwen3 only") or never synced from slime:

**Reverted from pre-vllm-project#126 (translated):**
- scripts/low_precision/run-qwen3-4b-fp8.sh
- scripts/low_precision/run-qwen3-30b-a3b-fp8.sh
- scripts/run-glm4-9B.sh
- scripts/run-moonlight-16B-A3B.sh
- scripts/run-qwen3-4B-base-sft.sh
- scripts/run-qwen3-32B.sh
- scripts/run-qwen3.5-35B-A3B-sft.sh

**New from slime@44d29ee (translated):**
- docs/en/get_started/agent.md
- examples/fully_async/run-qwen2.5-0.5B-fully_async.sh

All sglang engine flags translated to vllm equivalents (§2.4).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: unify pkill pattern to '[v]llm serve|VLL[M]::'

Standardize all scripts to use the bracket-escaped pkill pattern that
avoids matching pkill itself and also catches vLLM's renamed
subprocesses (VLLM::EngineCore, VLLM::Worker_TP*). Matches the
canonical pattern in command_utils.py.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* scripts: complete slime-exact translation of all 29 run scripts

Translate all slime scripts to vime following SGLANG_TO_VLLM_TRANSLATION.md:
- sglang→vllm prefix swap for CLI flags and variables
- _slime→_vime for checkpoint paths
- EP: --sglang-ep-size N → --vllm-enable-expert-parallel (boolean)
- Speculative: multi-param → --vllm-speculative-config JSON (§5.2)
- Delete genuinely sglang-coupled params (DP-attention, DeepEP, NSA, etc.)
- flashinfer → FLASHINFER case fix (§2.4)

23 new scripts + 6 existing updated to match slime@cutoff.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(scripts): correct model config source path in FP8 low_precision scripts

The FP8 scripts used `${SCRIPT_DIR}/../scripts/models/` which resolves
to `scripts/scripts/models/` (non-existent). Changed to `../models/`
to match the INT4 scripts. Same fix as slime PR #2094.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(gpt-oss): fused BF16 format, bridge API patch, bshd qkv format

Three fixes needed to run GPT-OSS 20B RLHF on vLLM backend:

1. hf_weight_iterator_bridge: match Megatron-Bridge 0.5.0 API
   _patch_bridge_expert_cache_to_cpu monkey-patches GPTOSSBridge.
   maybe_modify_converted_hf_weight gained a 4th `hf_state_dict`
   parameter; the patched wrapper only accepted 3, causing TypeError
   during weight sync.

2. run-gpt-oss-20B: point --hf-checkpoint at fused BF16 format
   vLLM's _load_weights_other expects gate_up_proj [E, hidden, 2*ffn]
   (fused). The old per-expert split format (experts.{e}.gate_proj.weight)
   causes KeyError on bias loading. Use tools/convert_gpt_oss_to_fused.py
   to convert an existing per-expert checkpoint, or re-run
   preprocess_gpt_oss.py to produce fused format directly.

3. run-gpt-oss-20B: add --qkv-format bshd + fix seq-length
   GPT-OSS uses learnable softmax (sink attention). TransformerEngine
   disables all attention backends when softmax_type=learnable and
   qkv_format=thd (packed sequences). --qkv-format bshd avoids this.
   --use-dynamic-batch-size is incompatible with bshd; replaced with
   fixed --seq-length 10240 (covers 8192 max response + prompt headroom).

tools/convert_gpt_oss_to_fused.py: new tool to convert per-expert BF16
checkpoint (output of old preprocess_gpt_oss.py) to the fused HF format
expected by vLLM without re-running the slow MXFP4 dequantization.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix(scripts): replace pkill -9 vllm with precise -f pattern (21 files)

pkill -9 vllm matches any process named "vllm" and can inadvertently
kill unrelated vllm processes (e.g. background services). Use the same
pattern as PR vllm-project#220 which targets only vllm serve and Ray VLL[M]:: actors:

  pkill -9 -f '[v]llm serve|VLL[M]::'

Also updates the inline form used in multi-node SSH worker restart
commands (run-qwen3-235B-A22B*.sh, run-qwen3.5-27B.sh, etc.).

Skipped: scripts/run-gpt-oss-20B.sh (uses pkill -9 -f "vllm serve" already),
scripts/run-minimax-m2.sh and run-glm4.7-*.sh (already used -f "vllm serve").

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* chore(scripts): remove run-qwen3-4B-amd.sh from this PR

AMD-specific script is out of scope for the gb300-complete-port PR.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* sync(docs+scripts): port docs/examples from slime-44d29ee, fix script translations

- Add missing EN/ZH docs: low-precision, on-policy-distillation, get_started/agent,
  pd-disaggregation (heterogeneous server groups fix), examples zh docs
- Add missing examples: on_policy_distillation, eval_multi_task, delta_weight_sync,
  geo3k images
- Fix vLLM flag translations across all example docs:
  - --vllm-mem-fraction-static → --vllm-gpu-memory-utilization
  - Remove non-existent dp-attention flags (--vllm-enable-dp-attention, --vllm-dp-size,
    --vllm-moe-dense-tp-size, --vllm-enable-dp-lm-head, --vllm-ep-size)
  - --vllm-ep-num-redundant-experts → --vllm-eplb-config
  - --vllm-cuda-graph-bs → --vllm-max-cudagraph-capture-size
  - sglang speculative flags → --vllm-speculative-config JSON
  - GLM-4.7 MTP: method=eagle → method=mtp, num_speculative_tokens=4 → 3
  - sgl-router → vllm-router; THUDM/vime → vllm-project/vime
- Fix scripts: restore run-kimi-k2-Instruct/Thinking/qwen3-4B/qwen3-235B-A22B to
  slime-44d29ee-as-vime + pkill precision fix only; restore int4 python3 path

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* revert(scripts): pkill -9 -f pattern back to pkill -9 vllm, align with slime

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* revert(pkill): align all remaining kill patterns with slime (pkill -9 vllm)

Covers examples/, docs/, tests/, and vime/utils -- previously missed in
the scripts/ revert.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* chore: remove gpt-oss-20B script and convert tool (moved to separate PR)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants