docker: rename published image from inferactinc/public to vllm/vime - #283
Merged
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request updates the Docker image repository from inferactinc/public to vllm/vime across the codebase, including Buildkite configurations, Docker build scripts, and documentation. The image tags have also been simplified by removing the vime- prefix. I have no feedback to provide as there are no review comments.
aoshen02
force-pushed
the
docker/rename-to-vllm-vime
branch
from
June 23, 2026 01:02
f895038 to
ccfdb87
Compare
Replace all references to `inferactinc/public:vime-*` with `vllm/vime:*` across CI, docs, and the release justfile. The image is now published under the official vllm DockerHub namespace (`vllm/vime:latest`, `vllm/vime:test-latest`) as a multi-arch manifest (amd64 + arm64). Signed-off-by: aoshen02 <aoshen@inferact.ai> Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
aoshen02
force-pushed
the
docker/rename-to-vllm-vime
branch
from
June 23, 2026 04:05
ccfdb87 to
4ea5f66
Compare
CalvinXKY
approved these changes
Jun 23, 2026
aoshen02
added a commit
that referenced
this pull request
Jun 25, 2026
Rebased onto upstream/main 7198547 (now includes #260 scripts port, #280 tau-bench agent_strategy, #283 docker image rename, #257 data fix). Effect vs prior base: the 12 script files #260 ported moved from new-to-vime to in-scope; 11 auto-merged clean, only scripts/run-deepseek-r1.sh newly conflicts (theirs adds /sgl-workspace dead path -> resolve by dropping). 3-way merge results on new base: 52 clean / 46 conflict (110 blocks) / 50 new-to-vime / 5 del Resolves PR #286 conflict-with-main. Conflict markers preserved for commit 2. SOP: knowledge/rl/slime-to-vime-sync-sop.md
aoshen02
added a commit
that referenced
this pull request
Jun 25, 2026
Mechanical commit 1 of 2 (per knowledge/rl/slime-to-vime-sync-sop.md §2). diff3 translated 3-way merge on upstream/main (incl #260/#280/#283/#257): ours = vime@main, base = translate(slime@#2013), theirs = translate(slime@#2125) Translation fixes vs prior attempt: - casing: SGLang->vLLM (prose) / SGLang<X>->VLLM<X> (identifiers); killed VLlm artifact (was 35 files) - dotted module refs slime.X->vime.X now translated (was leaking 'from slime.backends') These resolved 9 spurious conflicts (46->37 files). Results: 61 clean / 37 conflict (diff3 markers preserved) / 50 new-to-vime / 5 del. Conflict markers use readable -L labels (ours/base/theirs). Resolve in commit 2. Non-conflict provenance fix: vimerl/vime -> vllm/vime in 2 example docs. Engine patch handling (docker/patch/) deferred to commit 2 per SOP §4.5.
aoshen02
added a commit
that referenced
this pull request
Jun 26, 2026
Mechanical commit 1 of 2 (per knowledge/rl/slime-to-vime-sync-sop.md §2). diff3 translated 3-way merge on upstream/main (incl #260/#280/#283/#257): ours = vime@main, base = translate(slime@#2013), theirs = translate(slime@#2125) Translation fixes vs prior attempt: - casing: SGLang->vLLM (prose) / SGLang<X>->VLLM<X> (identifiers); killed VLlm artifact (was 35 files) - dotted module refs slime.X->vime.X now translated (was leaking 'from slime.backends') These resolved 9 spurious conflicts (46->37 files). Results: 61 clean / 37 conflict (diff3 markers preserved) / 50 new-to-vime / 5 del. Conflict markers use readable -L labels (ours/base/theirs). Resolve in commit 2. Non-conflict provenance fix: vimerl/vime -> vllm/vime in 2 example docs. Engine patch handling (docker/patch/) deferred to commit 2 per SOP §4.5. Signed-off-by: aoshen02 <aoshen@inferact.ai>
CalvinXKY
pushed a commit
that referenced
this pull request
Jun 29, 2026
…eview] (#286) * sync(slime #2014..#2125): diff3 3-way merge, conflicts preserved Mechanical commit 1 of 2 (per knowledge/rl/slime-to-vime-sync-sop.md §2). diff3 translated 3-way merge on upstream/main (incl #260/#280/#283/#257): ours = vime@main, base = translate(slime@#2013), theirs = translate(slime@#2125) Translation fixes vs prior attempt: - casing: SGLang->vLLM (prose) / SGLang<X>->VLLM<X> (identifiers); killed VLlm artifact (was 35 files) - dotted module refs slime.X->vime.X now translated (was leaking 'from slime.backends') These resolved 9 spurious conflicts (46->37 files). Results: 61 clean / 37 conflict (diff3 markers preserved) / 50 new-to-vime / 5 del. Conflict markers use readable -L labels (ours/base/theirs). Resolve in commit 2. Non-conflict provenance fix: vimerl/vime -> vllm/vime in 2 example docs. Engine patch handling (docker/patch/) deferred to commit 2 per SOP §4.5. Signed-off-by: aoshen02 <aoshen@inferact.ai> * sync(slime #2014..#2125): resolve all conflicts (commit 2) Resolved all 37 conflict files / 84 diff3 blocks per agent_run RESOLUTION_POLICY. Principle: keep vime vLLM impl (ours) + incorporate slime's new features (theirs). Highlights: - vLLM API form kept everywhere: /inference/v1/generate, choices parsing, AsyncEngineArgs, vLLM flag names (--vllm-gpu-memory-utilization etc). - Dropped all vllm.srt.* imports (non-existent in real vLLM). - Accepted new slime features: delta-weight-sync CLI args, append_response_tokens (Sample), get_server_info/start_external_rollout_servers/get_rollout_num_engines, old-router(<=0.2.1) compat, TrajectoryManager adapter design (vime already adopted it). - Kept vime-only: --rollout-external, add_router_arguments, _get_metrics_router_addr, reinit_wandb_primary_with_open_metrics, update_tracking_open_metrics, modal sandbox, VIME_AGENT_* env names, local-vLLM tau-bench user sim. - Engine patches (docker/patch/): kept ours vllm.patch (22-line MoE fix), dropped theirs sglang 2674-line content; deleted sglang-only vllm-top_p.patch (per SOP 4.5). - Dockerfile kept ours (vllm/vllm-openai base); version.txt accepted theirs nightly. - run-deepseek-r1.sh: dropped /sgl-workspace dead-path env. Deviations from policy (documented): vllm_rollout.py abort path kept ours pause/drain (abort_servers_until_idle would break partial-rollout drain + leave paused_workers unbound). README ecosystem section left empty (ours) pending de-translation of provenance. All changed .py py_compile clean; zero conflict markers; no sglang/slime leakage. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): pre-commit lint/format on resolved files - re-add dropped `from vllm_router.launch_router import RouterArgs` import in vime/utils/arguments.py (add_router_arguments uses it; F821 from conflict resolution) - drop unused base_top_p_token_ids/offsets in vllm_streaming_rollout.py (F841; came from theirs but ours's choices-parsing path doesn't use them) - black/isort autoformat (anthropic.py, test_agent/*, arguments.py) - pipeline.yml: agent tests moved to tests/test_agent/; wire new CPU tests pre-commit: all hooks pass (ruff/autoflake/isort/black/yaml). Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): make CPU CI green (engine import, docker build, agent/utils tests) Local CPU CI (pre-commit + plugin + agent + utils) all green on 8xH200 host in python:3.11 containers. Fixes found while running it: - vllm_engine.py: make 'import vllm_router'/'packaging.parse' lazy (inside _register_to_router); top-level import broke CPU import (vllm_router absent in CPU CI). The old-router(<=0.2.1) compat branch is theirs-accepted. - docker/Dockerfile: TMS_CUDA_MAJOR=12 for torch_memory_saver pin (its build backend now requires it for CUDA wheels; base is cu129). Unblocks image build. - agent/adapters/common.py: _run_turn called parse_model_output() without the required tokenizer= kwarg -> 500s in adapter tests. Pass tokenizer=tok. - tests/test_agent/_fakes.py: FakeVLLMServer served sglang /generate + meta_info; retarget to vime /inference/v1/generate + choices shape + x-session-id header. - tests/test_agent/test_adapters.py: parse_model_output(tokenizer=...) + assert vime body keys (token_ids/max_tokens). - tests/utils/test_vllm_config.py: vLLMConfig->VllmConfig (4 sites); fake router returns 3-tuple (ip,port,prom) matching _start_router; drop spurious resolve(). - tests/test_megatron_argument_validation.py: add num_gpus_per_node=8 to the vime_validate_args fixture (vime colocate override needs it). - .buildkite/pipeline.yml: agent tests -> tests/test_agent/*; +cispo_loss, +logprob_response_spans (CPU-safe); test_rollout_metrics stays GPU-only (imports vllm). Engine patch verdict (PATCH_ASSESSMENT.md): P1-P7 vLLM doesn't need (NIXL/Mooncake native); kept ours vllm.patch, dropped sglang content + top_p.patch. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): unblock GPU run (scipy pin, router arg dup, top-p-replay gate) Found running GPU CI on 8xH200 with vllm/vime:latest: - docker/Dockerfile: pin scipy<1.14 next to numpy<2 (scipy drifted to 1.18 which needs numpy>=2 and uses removed np.long -> 'import vllm' crash). - vllm_utils/arguments.py: drop sync-added RouterArgs.add_cli_args + its import in add_vllm_router_arguments. main exposes the full router surface only in utils.add_router_arguments; the duplicate re-registered --router-request-timeout-secs -> argparse conflict at train startup. - megatron_utils/loss.py: get_rollout_top_p_logprob_kwargs falls back to full-vocab logprob when top-p nucleus token ids are absent instead of raising. slime's top-p-replay needs engine-returned top-p tokens; vime's vLLM /inference/v1/generate does not expose them (sglang-only). Matches vime pre-sync behavior; flagged in OVERNIGHT_REPORT for review. Image import smoke + Megatron ckpt load + 4x VLLMEngine bringup + NCCL weight transfer all confirmed working in-image before this. Signed-off-by: aoshen02 <aoshen@inferact.ai> * ci(gpu): drop deleted test_qwen2.5_0.5B_ppo_critic_only_short from short suite slime deleted tests/test_qwen2.5_0.5B_ppo_critic_only_short.py this window (#2014..#2125); gpu_suites.py still listed it -> 'no such file' exit 2. Critic-only path is still covered by test_qwen3_4B_ppo_train_critic_only (megatron suite). Other 3 short tests (gsm8k_async, gsm8k, fully_async) pass on 8xH200. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): restore base_log_probs init in streaming rollout (None+list crash) GPU test_qwen3_4B_streaming_partial_rollout hit 'TypeError: unsupported operand +: NoneType and list' at vllm_streaming_rollout.py:234. The conflict resolution changed base_log_probs from main's `list(sample.rollout_log_probs or [])` to a None-able form; a fresh sample (rollout_log_probs=None) then did None + call_log_probs. Restored main's form. Real sync-resolution regression caught by GPU CI. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): streaming rollout uses append_response_tokens (was renamed update_from_meta_info) slime #2110 renamed Sample.update_from_meta_info -> append_response_tokens; vllm_rollout was updated but vllm_streaming_rollout still called the old name (AttributeError at generate_streaming). Streaming already accumulates tokens incrementally for partial-rollout, so call append_response_tokens(meta_info=meta) with tokens omitted -> metadata-only finalize (no double-append). Caught by GPU test_qwen3_4B_streaming_partial_rollout. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): restore vime unconditional colocate rollout_num_gpus re-derive 3-way compare (slime / vime-main / PR) showed the merge made a broken hybrid: it KEPT vime's num_gpus_per_node colocate override (which assumes rollout_num_gpus is forced to actor_num_gpus_per_node*actor_num_nodes) but REPLACED vime's unconditional re-derive (`!= -> re-derive`) with slime's `is None`-only form. When a colocate test's rollout_num_gpus is non-None but mismatches, it was left mis-sized -> engine/GPU misplacement -> mixed_offload IPC-UUID mismatch + ckpt 'Free memory < util'. slime passes (no override, self-consistent is-None); vime main passes (override + unconditional re-derive, coupled). Restore vime's re-derive; keep slime's new rollout_num_gpus==0 branch (checked first). Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): slime-consistency review pass (docs, rollout routed_experts, comments) Docs (EN+zh parity): - fault-tolerance: revert junk 'server'->'engine' mistranslation (keep correct /health endpoint, verified vs vllm_engine.py) - vllm-config: 'ServerArgs'->'EngineArgs' (sglang class -> vLLM AsyncEngineArgs u FrontendArgs); fix duplicated 'vllm-router (vllm-router)' alias - customization: restore over-deleted 'custom_generate -> list[Sample]' section + signature (dropped only the vime-absent search-r1 example link) Rollout: - routed_experts now flows through the slime-identical Sample._apply_meta_info (single assignment site, torch.int32 tensor matching downstream) instead of an inline numpy assign; vLLM .npy-on-choice decode stays (engine wire-format delta). Both vllm_rollout and vllm_streaming_rollout. Comments for future syncers: - --opd-teacher-model + on_policy_distillation: engine-driven divergence (vLLM model field; sglang /generate has none) - overrides / _vllm_server_field_names: AsyncEngineArgs u FrontendArgs == slime's sglang ServerArgs Examples/docker/etc: drop vime-absent npu/retool/search-r1/tau-bench files; restore eval_multi_task; docker alignment with slime. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(ci): pre-commit green — define base in streaming MM render (F821) + isort/black on test_agent Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(ci): correct two mistranslated CPU tests (colocate rollout-gpu re-derive; parse_model_output tokenizer) - test_..._preserves_larger_rollout_gpus_under_colocate asserted slime behavior (==12); vime re-derives to actor*nodes=8 under colocate (commit 9701304). Renamed + assert ==8 + divergence note. vime-main never had the test; slime does. - test_parse_model_output_plain_text_no_parsers called parse_model_output without the required tokenizer kwarg (#198 made it required for vLLM parsers). Pass tokenizer=None (unused on the no-parser path). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): streaming rollout posts to /inference/v1/generate, not sglang /generate The slime diff3 merge took slime's sglang endpoint (/generate) for the streaming rollout POST instead of keeping vime's vLLM endpoint (/inference/v1/generate). main (7198547) had the correct URL; the sync regressed it (and dropped the base var). Result: 404 Not Found at vllm_streaming_rollout.py:182 -> test_qwen3_4B_streaming_partial_rollout fails. Caught on a clean h200 node. The file's own docstrings already say /inference/v1/generate throughout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): thread rollout port cursor globally across multi-model engines The diff3 merge adopted slime's deferred rollout-engine init (start_rollout_servers now returns pending_init_handles awaited by the caller instead of ray.get-ing each model's engines before the next). That broke an implicit invariant the per-model port_cursors reset relied on: in main, model 0's engines were fully bound before model 1 allocated ports, so the free-port bind-test in _allocate_rollout_engine_addr_and_ports_normal skipped model 0's ports. With deferred init, model 1 allocates while model 0 is unbound, the bind-test sees the base ports free, and a second model (e.g. mixed_offload's frozen "ref") lands on the same 15000-15003 as the actor. The actor's POST /update_weights to :15002 then hits the never-started ref engine -> vLLM 500 "start_weight_update must be called before update_weights" (test_vllm_config_mixed_offload[_ft]). Fix: initialize port_cursors once before the model loop so the per-node next-free cursor is monotonic across all models, keeping every engine's ports disjoint regardless of bind timing. Single-model behaviour is unchanged; the per-model reset only existed to scope cursors that are already node-keyed. Caught on h200 GPU CI (new nightly-dev-20260618a image, which added the start_weight_update-before-update_weights enforcement that exposed the collision). Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): restore robust pkill pattern so ckpt cleanup kills vLLM ray actors PR #260 ("complete slime-exact port") changed execute_train's pre-launch cleanup from pkill -9 -f '[v]llm serve|VLL[M]::' to pkill -9 vllm as an over-literal sglang->vllm translation. But the vLLM rollout engine runs as Ray actor processes whose process *name* is python/ray, with "VLLM::" only in the command line — so `pkill -9 vllm` (name match, no -f) does not kill them. Leftover engine processes from the ckpt test's save phase survive into the load phase, holding ~115 GiB, so the load-phase engine starts with ~24/139 GiB free and dies with "Free memory ... less than desired GPU memory utilization (0.8, 111.84 GiB)" (test_qwen3_4B_ckpt.py, both --async-save and not). Restore the cmdline-match pattern `-f '[v]llm serve|VLL[M]::'`. Bisected on h200: ckpt PASSES at 289ee6d / e62d44f (old pattern, 4/4 runs) and FAILS at 7198547/main + PR (new pattern, 0/3), same old image -> code regression in #260. Verified: pkill-fixed PR code + new pr286 image -> ckpt PASS (579s). Signed-off-by: aoshen02 <aoshen@inferact.ai> * chore(sync): replace all `pkill -9 vllm` with cmdline-match pattern Same root cause as 764e1e1 (command_utils.py): `pkill -9 vllm` matches by process *name*, but vLLM rollout engines run as Ray actor processes (python/ray named, "VLLM::" only in the command line), so the name match never kills them. Apply the robust cmdline pattern `pkill -9 -f '[v]llm serve|VLL[M]::'` everywhere the bare `pkill -9 vllm` cleanup was used across run/example scripts, so leftover engines don't squat GPUs across runs. No logic change beyond the kill pattern. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(docker): restore vLLM core.py partial-wake sleep-guard (#44483) Commit d41f0aa removed the core.py sleep-guard hunk from docker/patch/latest/vllm.patch on the assumption it was "already in v0.23.0". It is not: stock v0.23.0 `vllm/v1/engine/core.py` calls `resume_scheduler()` and `execute_dummy_batch()` even during a partial (weights-only) wake. So a colocate DP+EP pd_mooncake rollout, right after `POST /wake_up?tags=weights` (KV cache still released under level-2 sleep), has its DP busy-loop fire a decode-shaped dummy batch that touches freed KV -> the scheduler_metadata write in flashattn_mla.py:234 (MLA, glm4.7) and flash_attn.py:547 (FA3, qwen3.6) raises `CUDA error: invalid argument`. Restore the guard (`if not self.model_executor.is_sleeping` around resume_scheduler; `if not self.is_sleeping()` around execute_dummy_batch), keeping the all2all_utils weight-reload fix. This is the #173 sleep-guard patch re-expressed against v0.23.0 line numbers. Verified: git-apply --check clean against stock v0.23.0; both guards land; glm4.7/qwen3.6 pd_mooncake reproduced the crash without it (the 8-day-old image that still carried the guard passes both). Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(docker): drop scipy<1.14 pin, mirror slime numpy<2 only The scipy<1.14 pin (added in 0cda11c while chasing the pd_mooncake crash) was a red herring: the real cause was the dropped core.py partial-wake sleep-guard, now restored. slime-2125-as-vime pins only `numpy<2`; this restores that exact line. numpy 1.26.4 + scipy resolved naturally matches the working baseline. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(docker): keep FlashQLA install gated behind INSTALL_FLASHQLA=0 slime installs FlashQLA unconditionally, but it is sm90/Hopper-only. Restore vime's original gated form (default off; --qwen-gdn-backend fla elsewhere). CI build passes --build-arg INSTALL_FLASHQLA=1 to include it. Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs(vllm-config): fix inference-only FAQ — vime launches engines in-process The mechanical mirror of slime's answer steered users to vLLM's standalone `vllm serve` (slime's `launch_server` analog) for inference-only. That is misleading for vime: like slime, vime launches the vLLM engines in-process from `--vllm-config` (same in-process path as training), so a rollout-only run serves directly with no separate server process. Point standalone users to `--rollout-external-engine-addrs` instead. EN + ZH. Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs(debug): restore INT4 / Compressed-Tensors checkpoint section vime's debug.md was missing slime's "INT4 / Compressed-Tensors Quantization Checkpoint Issues" section (slime #1642) — dropped in an earlier sync, not present on main. Restore it (EN + ZH), translated sglang→vLLM / Megatron→vLLM. Covers the quantization_config.ignore list, all-zero MoE router weights (mlp.gate.weight) when mis-quantized, missing safetensors shards, and diagnosis via --check-weight-update-equal / --debug-rollout-only. Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs(vllm-config): use _run_vllm_server for inference-only FAQ Keep slime's wording; the only engine-coupled fix is the standalone launcher name. slime's `launch_server` is its in-process engine entry; vime's analog is `_run_vllm_server` (vllm_engine.py, launched via multiprocessing.Process), not the standalone `vllm serve` CLI. EN + ZH. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(docker): restore scipy pin (scipy<1.18) — vime base needs it Reverts the scipy-pin removal in b359b24, which was wrong: vime's vllm/vllm-openai base ships no scipy, so unpinned the build pulls scipy>=1.18, which hard-requires numpy>=2 and uses np.long (removed numpy>=1.24) -> crashes against the numpy<2 reinstall (Megatron needs numpy 1.x). slime's sglang base resolves scipy 1.17.1 natively (numpy-1.x compatible), so slime needs no pin; this is a base-image divergence, not a red herring. Pin boundary is 1.18 (slime runs 1.17.1), not the earlier 1.14 over-estimate. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(scripts): properly translate sglang args in glm5.2-744B + glm4.7-355B-delta These two were prefix-swapped (--sglang-X -> --vllm-X) without semantic mapping, leaving ~20 args that aren't vLLM AsyncEngineArgs (argparse would reject). Apply the knowledge/rl/sglang-to-vllm-translation.md §5.5 mappings: - dp-size->data-parallel-size, ep-size->enable-expert-parallel, max-running-requests-> max-num-seqs, cuda-graph-max-bs->max-cudagraph-capture-size - 5x/4x --speculative-* -> one --vllm-speculative-config JSON (§5.2) - DeepEP: per-group deepep_mode auto/low_latency -> all2all_backend deepep_high_throughput/ low_latency in the --vllm-config overrides (vLLM has no 'auto'; PD encodes it per-role) - watchdog-timeout -> env VLLM_ENGINE_ITERATION_TIMEOUT_S - drop sglang-only: dp-attention / dp-lm-head / moe-dense-tp / disable-overlap-schedule / NSA backends (vLLM selects DeepSeek sparse attn per model) / engine delta-receiver knobs - flag PD mooncake transport (-> --vllm-kv-transfer-config) as fabric-specific TODO These are 744B/355B scripts not runnable in CI — translations are SOP-mapped but hardware-unvalidated (flagged inline). Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(args): hard-guard unverified delta weight-sync mode --update-weight-mode=delta (PR #278 lineage) is not yet validated on vime+vLLM (vLLM exposes dense/sparse_flat only, not slime's gap-delta/ zstd encoding). Raise NotImplementedError at arg-validation so it fails fast at startup instead of crashing mid weight-sync. Downstream delta code is kept untouched; remove this raise once a real delta-load run passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs,test: correct rollout engine endpoint to /inference/v1/generate The agent rollout engine is reached at vime's vLLM ``/inference/v1/generate`` (see vllm_rollout.get_model_url default + common.call_vllm_generate), not the bare ``/generate`` of sglang. Fix the imprecise path in adapter/test docstrings and comments, and rewrite the vllm-config.md custom-rollout examples (en+zh): they were still sglang-shaped (``/generate`` path + ``{"text":..., "return_logprob": True}`` body). Use vime's real request schema instead -- ``{"model","token_ids", "sampling_params"}`` with ``max_tokens``/``logprobs``, ``prompt_logprobs`` for fixed-sequence scoring, and the ``choices[0]`` response shape. No code/logic change: comments, docstrings, and doc examples only. The Megatron training server's own ``/generate`` endpoint and the sglang citation in arguments.py are correct and left untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * style(args): collapse delta-guard message to one line (black) The delta-guard NotImplementedError message was split across two adjacent string literals; black on the CI (line-length 119) collapses/normalizes it. Make it a single clean literal so pre-commit is green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * test(args): assert delta weight-sync is guarded off (not per-condition) The hard delta guard (7bb19e6) raises NotImplementedError at the top of the delta branch, making the downstream colocate / unknown-transport rejections unreachable. Replace test_update_weight_delta_rejects_colocate and test_update_weight_delta_rejects_unknown_transport (whose ValueError paths no longer fire) with a single test_update_weight_delta_disabled that asserts the guard raises for any delta config. Breadcrumb left to restore the per-condition tests when delta is verified and the guard is lifted. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): restore dropped weight-sync metrics chain + delta dispatch (slime parity) A mirror cross-check of the delta-weight-sync surface (slime #1806/#1991) found the #2014..#2125 sync had silently dropped several slime-faithful pieces: * extra_metrics logging chain — slime threads weight-update metrics from the actor through log_perf_data -> log_perf_data_raw. vime dropped the param at all three layers, so weight-update metrics were never logged. Restored: train_metric_utils.log_perf_data_raw(extra_metrics=...), data.log_perf_data passthrough, and actor passing self.weight_updater.pop_metrics(). * pop_metrics on UpdateWeightFromDistributed — the default (non-colocate, nccl) weight_updater. slime gives all three updaters a pop_metrics() stub so the actor can call it uniformly; vime kept it on tensor/disk but dropped it on distributed, which would AttributeError once the actor calls it. Restored the ~5-line stub (delta-specific plumbing stays dropped — vime+vLLM has no DeltaSpec). * actor delta-mode dispatch branch — restores the elif selecting UpdateWeightFromDistributedDelta. Dead code behind the validation guard that rejects --update-weight-mode=delta, so vime mirrors slime with the guard as the single divergence. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * chore(sync): align comments to the mechanical mirror Comment-only pass; no behavior change. Aligns vime comments to what a faithful slime->vime translation would carry: * Strip vime-divergence rationale markers (delta guard, opd-teacher-model, top-p fallback, colocate/delta tests). The rationale belongs in the divergence manifest, not inline; the guarded code + self-explanatory NotImplementedError messages stand on their own. * De-verbose vLLM-specific comments to mirror scale: the router-args block, the AsyncEngineArgs u FrontendArgs docstring, and the MoE-replay / streaming-rollout blocks that slime does not carry at that length. * Restore slime-original comments the sync had dropped or naively translated, with judgment translation of sglang-specific terms: session_id routing ("vLLM router", not the mechanical "Model Gateway"), "Prepare payload for vLLM server", the unique-session_id loop, and the pending-tasks wait. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs(delta): note delta weight-sync not yet verified on vime+vLLM (PR #286 review) Per review on PR #286: delta weight sync is documented here but the arg guard disables --update-weight-mode=delta. Add a top-of-page note (en + zh) so users see it before hitting NotImplementedError at argparse. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> --------- Signed-off-by: aoshen02 <aoshen@inferact.ai> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
aoshen02
added a commit
to aoshen02/vime
that referenced
this pull request
Jul 15, 2026
…llm-project#283) Replace all references to `inferactinc/public:vime-*` with `vllm/vime:*` across CI, docs, and the release justfile. The image is now published under the official vllm DockerHub namespace (`vllm/vime:latest`, `vllm/vime:test-latest`) as a multi-arch manifest (amd64 + arm64). Signed-off-by: aoshen02 <aoshen@inferact.ai> Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
aoshen02
added a commit
to aoshen02/vime
that referenced
this pull request
Jul 15, 2026
…eview] (vllm-project#286) * sync(slime #2014..#2125): diff3 3-way merge, conflicts preserved Mechanical commit 1 of 2 (per knowledge/rl/slime-to-vime-sync-sop.md §2). diff3 translated 3-way merge on upstream/main (incl vllm-project#260/vllm-project#280/vllm-project#283/vllm-project#257): ours = vime@main, base = translate(slime@#2013), theirs = translate(slime@#2125) Translation fixes vs prior attempt: - casing: SGLang->vLLM (prose) / SGLang<X>->VLLM<X> (identifiers); killed VLlm artifact (was 35 files) - dotted module refs slime.X->vime.X now translated (was leaking 'from slime.backends') These resolved 9 spurious conflicts (46->37 files). Results: 61 clean / 37 conflict (diff3 markers preserved) / 50 new-to-vime / 5 del. Conflict markers use readable -L labels (ours/base/theirs). Resolve in commit 2. Non-conflict provenance fix: vimerl/vime -> vllm/vime in 2 example docs. Engine patch handling (docker/patch/) deferred to commit 2 per SOP §4.5. Signed-off-by: aoshen02 <aoshen@inferact.ai> * sync(slime #2014..#2125): resolve all conflicts (commit 2) Resolved all 37 conflict files / 84 diff3 blocks per agent_run RESOLUTION_POLICY. Principle: keep vime vLLM impl (ours) + incorporate slime's new features (theirs). Highlights: - vLLM API form kept everywhere: /inference/v1/generate, choices parsing, AsyncEngineArgs, vLLM flag names (--vllm-gpu-memory-utilization etc). - Dropped all vllm.srt.* imports (non-existent in real vLLM). - Accepted new slime features: delta-weight-sync CLI args, append_response_tokens (Sample), get_server_info/start_external_rollout_servers/get_rollout_num_engines, old-router(<=0.2.1) compat, TrajectoryManager adapter design (vime already adopted it). - Kept vime-only: --rollout-external, add_router_arguments, _get_metrics_router_addr, reinit_wandb_primary_with_open_metrics, update_tracking_open_metrics, modal sandbox, VIME_AGENT_* env names, local-vLLM tau-bench user sim. - Engine patches (docker/patch/): kept ours vllm.patch (22-line MoE fix), dropped theirs sglang 2674-line content; deleted sglang-only vllm-top_p.patch (per SOP 4.5). - Dockerfile kept ours (vllm/vllm-openai base); version.txt accepted theirs nightly. - run-deepseek-r1.sh: dropped /sgl-workspace dead-path env. Deviations from policy (documented): vllm_rollout.py abort path kept ours pause/drain (abort_servers_until_idle would break partial-rollout drain + leave paused_workers unbound). README ecosystem section left empty (ours) pending de-translation of provenance. All changed .py py_compile clean; zero conflict markers; no sglang/slime leakage. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): pre-commit lint/format on resolved files - re-add dropped `from vllm_router.launch_router import RouterArgs` import in vime/utils/arguments.py (add_router_arguments uses it; F821 from conflict resolution) - drop unused base_top_p_token_ids/offsets in vllm_streaming_rollout.py (F841; came from theirs but ours's choices-parsing path doesn't use them) - black/isort autoformat (anthropic.py, test_agent/*, arguments.py) - pipeline.yml: agent tests moved to tests/test_agent/; wire new CPU tests pre-commit: all hooks pass (ruff/autoflake/isort/black/yaml). Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): make CPU CI green (engine import, docker build, agent/utils tests) Local CPU CI (pre-commit + plugin + agent + utils) all green on 8xH200 host in python:3.11 containers. Fixes found while running it: - vllm_engine.py: make 'import vllm_router'/'packaging.parse' lazy (inside _register_to_router); top-level import broke CPU import (vllm_router absent in CPU CI). The old-router(<=0.2.1) compat branch is theirs-accepted. - docker/Dockerfile: TMS_CUDA_MAJOR=12 for torch_memory_saver pin (its build backend now requires it for CUDA wheels; base is cu129). Unblocks image build. - agent/adapters/common.py: _run_turn called parse_model_output() without the required tokenizer= kwarg -> 500s in adapter tests. Pass tokenizer=tok. - tests/test_agent/_fakes.py: FakeVLLMServer served sglang /generate + meta_info; retarget to vime /inference/v1/generate + choices shape + x-session-id header. - tests/test_agent/test_adapters.py: parse_model_output(tokenizer=...) + assert vime body keys (token_ids/max_tokens). - tests/utils/test_vllm_config.py: vLLMConfig->VllmConfig (4 sites); fake router returns 3-tuple (ip,port,prom) matching _start_router; drop spurious resolve(). - tests/test_megatron_argument_validation.py: add num_gpus_per_node=8 to the vime_validate_args fixture (vime colocate override needs it). - .buildkite/pipeline.yml: agent tests -> tests/test_agent/*; +cispo_loss, +logprob_response_spans (CPU-safe); test_rollout_metrics stays GPU-only (imports vllm). Engine patch verdict (PATCH_ASSESSMENT.md): P1-P7 vLLM doesn't need (NIXL/Mooncake native); kept ours vllm.patch, dropped sglang content + top_p.patch. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): unblock GPU run (scipy pin, router arg dup, top-p-replay gate) Found running GPU CI on 8xH200 with vllm/vime:latest: - docker/Dockerfile: pin scipy<1.14 next to numpy<2 (scipy drifted to 1.18 which needs numpy>=2 and uses removed np.long -> 'import vllm' crash). - vllm_utils/arguments.py: drop sync-added RouterArgs.add_cli_args + its import in add_vllm_router_arguments. main exposes the full router surface only in utils.add_router_arguments; the duplicate re-registered --router-request-timeout-secs -> argparse conflict at train startup. - megatron_utils/loss.py: get_rollout_top_p_logprob_kwargs falls back to full-vocab logprob when top-p nucleus token ids are absent instead of raising. slime's top-p-replay needs engine-returned top-p tokens; vime's vLLM /inference/v1/generate does not expose them (sglang-only). Matches vime pre-sync behavior; flagged in OVERNIGHT_REPORT for review. Image import smoke + Megatron ckpt load + 4x VLLMEngine bringup + NCCL weight transfer all confirmed working in-image before this. Signed-off-by: aoshen02 <aoshen@inferact.ai> * ci(gpu): drop deleted test_qwen2.5_0.5B_ppo_critic_only_short from short suite slime deleted tests/test_qwen2.5_0.5B_ppo_critic_only_short.py this window (#2014..#2125); gpu_suites.py still listed it -> 'no such file' exit 2. Critic-only path is still covered by test_qwen3_4B_ppo_train_critic_only (megatron suite). Other 3 short tests (gsm8k_async, gsm8k, fully_async) pass on 8xH200. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): restore base_log_probs init in streaming rollout (None+list crash) GPU test_qwen3_4B_streaming_partial_rollout hit 'TypeError: unsupported operand +: NoneType and list' at vllm_streaming_rollout.py:234. The conflict resolution changed base_log_probs from main's `list(sample.rollout_log_probs or [])` to a None-able form; a fresh sample (rollout_log_probs=None) then did None + call_log_probs. Restored main's form. Real sync-resolution regression caught by GPU CI. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): streaming rollout uses append_response_tokens (was renamed update_from_meta_info) slime #2110 renamed Sample.update_from_meta_info -> append_response_tokens; vllm_rollout was updated but vllm_streaming_rollout still called the old name (AttributeError at generate_streaming). Streaming already accumulates tokens incrementally for partial-rollout, so call append_response_tokens(meta_info=meta) with tokens omitted -> metadata-only finalize (no double-append). Caught by GPU test_qwen3_4B_streaming_partial_rollout. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): restore vime unconditional colocate rollout_num_gpus re-derive 3-way compare (slime / vime-main / PR) showed the merge made a broken hybrid: it KEPT vime's num_gpus_per_node colocate override (which assumes rollout_num_gpus is forced to actor_num_gpus_per_node*actor_num_nodes) but REPLACED vime's unconditional re-derive (`!= -> re-derive`) with slime's `is None`-only form. When a colocate test's rollout_num_gpus is non-None but mismatches, it was left mis-sized -> engine/GPU misplacement -> mixed_offload IPC-UUID mismatch + ckpt 'Free memory < util'. slime passes (no override, self-consistent is-None); vime main passes (override + unconditional re-derive, coupled). Restore vime's re-derive; keep slime's new rollout_num_gpus==0 branch (checked first). Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): slime-consistency review pass (docs, rollout routed_experts, comments) Docs (EN+zh parity): - fault-tolerance: revert junk 'server'->'engine' mistranslation (keep correct /health endpoint, verified vs vllm_engine.py) - vllm-config: 'ServerArgs'->'EngineArgs' (sglang class -> vLLM AsyncEngineArgs u FrontendArgs); fix duplicated 'vllm-router (vllm-router)' alias - customization: restore over-deleted 'custom_generate -> list[Sample]' section + signature (dropped only the vime-absent search-r1 example link) Rollout: - routed_experts now flows through the slime-identical Sample._apply_meta_info (single assignment site, torch.int32 tensor matching downstream) instead of an inline numpy assign; vLLM .npy-on-choice decode stays (engine wire-format delta). Both vllm_rollout and vllm_streaming_rollout. Comments for future syncers: - --opd-teacher-model + on_policy_distillation: engine-driven divergence (vLLM model field; sglang /generate has none) - overrides / _vllm_server_field_names: AsyncEngineArgs u FrontendArgs == slime's sglang ServerArgs Examples/docker/etc: drop vime-absent npu/retool/search-r1/tau-bench files; restore eval_multi_task; docker alignment with slime. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(ci): pre-commit green — define base in streaming MM render (F821) + isort/black on test_agent Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(ci): correct two mistranslated CPU tests (colocate rollout-gpu re-derive; parse_model_output tokenizer) - test_..._preserves_larger_rollout_gpus_under_colocate asserted slime behavior (==12); vime re-derives to actor*nodes=8 under colocate (commit 9701304). Renamed + assert ==8 + divergence note. vime-main never had the test; slime does. - test_parse_model_output_plain_text_no_parsers called parse_model_output without the required tokenizer kwarg (vllm-project#198 made it required for vLLM parsers). Pass tokenizer=None (unused on the no-parser path). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): streaming rollout posts to /inference/v1/generate, not sglang /generate The slime diff3 merge took slime's sglang endpoint (/generate) for the streaming rollout POST instead of keeping vime's vLLM endpoint (/inference/v1/generate). main (7198547) had the correct URL; the sync regressed it (and dropped the base var). Result: 404 Not Found at vllm_streaming_rollout.py:182 -> test_qwen3_4B_streaming_partial_rollout fails. Caught on a clean h200 node. The file's own docstrings already say /inference/v1/generate throughout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): thread rollout port cursor globally across multi-model engines The diff3 merge adopted slime's deferred rollout-engine init (start_rollout_servers now returns pending_init_handles awaited by the caller instead of ray.get-ing each model's engines before the next). That broke an implicit invariant the per-model port_cursors reset relied on: in main, model 0's engines were fully bound before model 1 allocated ports, so the free-port bind-test in _allocate_rollout_engine_addr_and_ports_normal skipped model 0's ports. With deferred init, model 1 allocates while model 0 is unbound, the bind-test sees the base ports free, and a second model (e.g. mixed_offload's frozen "ref") lands on the same 15000-15003 as the actor. The actor's POST /update_weights to :15002 then hits the never-started ref engine -> vLLM 500 "start_weight_update must be called before update_weights" (test_vllm_config_mixed_offload[_ft]). Fix: initialize port_cursors once before the model loop so the per-node next-free cursor is monotonic across all models, keeping every engine's ports disjoint regardless of bind timing. Single-model behaviour is unchanged; the per-model reset only existed to scope cursors that are already node-keyed. Caught on h200 GPU CI (new nightly-dev-20260618a image, which added the start_weight_update-before-update_weights enforcement that exposed the collision). Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): restore robust pkill pattern so ckpt cleanup kills vLLM ray actors PR vllm-project#260 ("complete slime-exact port") changed execute_train's pre-launch cleanup from pkill -9 -f '[v]llm serve|VLL[M]::' to pkill -9 vllm as an over-literal sglang->vllm translation. But the vLLM rollout engine runs as Ray actor processes whose process *name* is python/ray, with "VLLM::" only in the command line — so `pkill -9 vllm` (name match, no -f) does not kill them. Leftover engine processes from the ckpt test's save phase survive into the load phase, holding ~115 GiB, so the load-phase engine starts with ~24/139 GiB free and dies with "Free memory ... less than desired GPU memory utilization (0.8, 111.84 GiB)" (test_qwen3_4B_ckpt.py, both --async-save and not). Restore the cmdline-match pattern `-f '[v]llm serve|VLL[M]::'`. Bisected on h200: ckpt PASSES at 289ee6d / e62d44f (old pattern, 4/4 runs) and FAILS at 7198547/main + PR (new pattern, 0/3), same old image -> code regression in vllm-project#260. Verified: pkill-fixed PR code + new pr286 image -> ckpt PASS (579s). Signed-off-by: aoshen02 <aoshen@inferact.ai> * chore(sync): replace all `pkill -9 vllm` with cmdline-match pattern Same root cause as 764e1e1 (command_utils.py): `pkill -9 vllm` matches by process *name*, but vLLM rollout engines run as Ray actor processes (python/ray named, "VLLM::" only in the command line), so the name match never kills them. Apply the robust cmdline pattern `pkill -9 -f '[v]llm serve|VLL[M]::'` everywhere the bare `pkill -9 vllm` cleanup was used across run/example scripts, so leftover engines don't squat GPUs across runs. No logic change beyond the kill pattern. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(docker): restore vLLM core.py partial-wake sleep-guard (#44483) Commit d41f0aa removed the core.py sleep-guard hunk from docker/patch/latest/vllm.patch on the assumption it was "already in v0.23.0". It is not: stock v0.23.0 `vllm/v1/engine/core.py` calls `resume_scheduler()` and `execute_dummy_batch()` even during a partial (weights-only) wake. So a colocate DP+EP pd_mooncake rollout, right after `POST /wake_up?tags=weights` (KV cache still released under level-2 sleep), has its DP busy-loop fire a decode-shaped dummy batch that touches freed KV -> the scheduler_metadata write in flashattn_mla.py:234 (MLA, glm4.7) and flash_attn.py:547 (FA3, qwen3.6) raises `CUDA error: invalid argument`. Restore the guard (`if not self.model_executor.is_sleeping` around resume_scheduler; `if not self.is_sleeping()` around execute_dummy_batch), keeping the all2all_utils weight-reload fix. This is the vllm-project#173 sleep-guard patch re-expressed against v0.23.0 line numbers. Verified: git-apply --check clean against stock v0.23.0; both guards land; glm4.7/qwen3.6 pd_mooncake reproduced the crash without it (the 8-day-old image that still carried the guard passes both). Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(docker): drop scipy<1.14 pin, mirror slime numpy<2 only The scipy<1.14 pin (added in 0cda11c while chasing the pd_mooncake crash) was a red herring: the real cause was the dropped core.py partial-wake sleep-guard, now restored. slime-2125-as-vime pins only `numpy<2`; this restores that exact line. numpy 1.26.4 + scipy resolved naturally matches the working baseline. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(docker): keep FlashQLA install gated behind INSTALL_FLASHQLA=0 slime installs FlashQLA unconditionally, but it is sm90/Hopper-only. Restore vime's original gated form (default off; --qwen-gdn-backend fla elsewhere). CI build passes --build-arg INSTALL_FLASHQLA=1 to include it. Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs(vllm-config): fix inference-only FAQ — vime launches engines in-process The mechanical mirror of slime's answer steered users to vLLM's standalone `vllm serve` (slime's `launch_server` analog) for inference-only. That is misleading for vime: like slime, vime launches the vLLM engines in-process from `--vllm-config` (same in-process path as training), so a rollout-only run serves directly with no separate server process. Point standalone users to `--rollout-external-engine-addrs` instead. EN + ZH. Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs(debug): restore INT4 / Compressed-Tensors checkpoint section vime's debug.md was missing slime's "INT4 / Compressed-Tensors Quantization Checkpoint Issues" section (slime #1642) — dropped in an earlier sync, not present on main. Restore it (EN + ZH), translated sglang→vLLM / Megatron→vLLM. Covers the quantization_config.ignore list, all-zero MoE router weights (mlp.gate.weight) when mis-quantized, missing safetensors shards, and diagnosis via --check-weight-update-equal / --debug-rollout-only. Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs(vllm-config): use _run_vllm_server for inference-only FAQ Keep slime's wording; the only engine-coupled fix is the standalone launcher name. slime's `launch_server` is its in-process engine entry; vime's analog is `_run_vllm_server` (vllm_engine.py, launched via multiprocessing.Process), not the standalone `vllm serve` CLI. EN + ZH. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(docker): restore scipy pin (scipy<1.18) — vime base needs it Reverts the scipy-pin removal in b359b24, which was wrong: vime's vllm/vllm-openai base ships no scipy, so unpinned the build pulls scipy>=1.18, which hard-requires numpy>=2 and uses np.long (removed numpy>=1.24) -> crashes against the numpy<2 reinstall (Megatron needs numpy 1.x). slime's sglang base resolves scipy 1.17.1 natively (numpy-1.x compatible), so slime needs no pin; this is a base-image divergence, not a red herring. Pin boundary is 1.18 (slime runs 1.17.1), not the earlier 1.14 over-estimate. Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(scripts): properly translate sglang args in glm5.2-744B + glm4.7-355B-delta These two were prefix-swapped (--sglang-X -> --vllm-X) without semantic mapping, leaving ~20 args that aren't vLLM AsyncEngineArgs (argparse would reject). Apply the knowledge/rl/sglang-to-vllm-translation.md §5.5 mappings: - dp-size->data-parallel-size, ep-size->enable-expert-parallel, max-running-requests-> max-num-seqs, cuda-graph-max-bs->max-cudagraph-capture-size - 5x/4x --speculative-* -> one --vllm-speculative-config JSON (§5.2) - DeepEP: per-group deepep_mode auto/low_latency -> all2all_backend deepep_high_throughput/ low_latency in the --vllm-config overrides (vLLM has no 'auto'; PD encodes it per-role) - watchdog-timeout -> env VLLM_ENGINE_ITERATION_TIMEOUT_S - drop sglang-only: dp-attention / dp-lm-head / moe-dense-tp / disable-overlap-schedule / NSA backends (vLLM selects DeepSeek sparse attn per model) / engine delta-receiver knobs - flag PD mooncake transport (-> --vllm-kv-transfer-config) as fabric-specific TODO These are 744B/355B scripts not runnable in CI — translations are SOP-mapped but hardware-unvalidated (flagged inline). Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(args): hard-guard unverified delta weight-sync mode --update-weight-mode=delta (PR vllm-project#278 lineage) is not yet validated on vime+vLLM (vLLM exposes dense/sparse_flat only, not slime's gap-delta/ zstd encoding). Raise NotImplementedError at arg-validation so it fails fast at startup instead of crashing mid weight-sync. Downstream delta code is kept untouched; remove this raise once a real delta-load run passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs,test: correct rollout engine endpoint to /inference/v1/generate The agent rollout engine is reached at vime's vLLM ``/inference/v1/generate`` (see vllm_rollout.get_model_url default + common.call_vllm_generate), not the bare ``/generate`` of sglang. Fix the imprecise path in adapter/test docstrings and comments, and rewrite the vllm-config.md custom-rollout examples (en+zh): they were still sglang-shaped (``/generate`` path + ``{"text":..., "return_logprob": True}`` body). Use vime's real request schema instead -- ``{"model","token_ids", "sampling_params"}`` with ``max_tokens``/``logprobs``, ``prompt_logprobs`` for fixed-sequence scoring, and the ``choices[0]`` response shape. No code/logic change: comments, docstrings, and doc examples only. The Megatron training server's own ``/generate`` endpoint and the sglang citation in arguments.py are correct and left untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * style(args): collapse delta-guard message to one line (black) The delta-guard NotImplementedError message was split across two adjacent string literals; black on the CI (line-length 119) collapses/normalizes it. Make it a single clean literal so pre-commit is green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * test(args): assert delta weight-sync is guarded off (not per-condition) The hard delta guard (7bb19e6) raises NotImplementedError at the top of the delta branch, making the downstream colocate / unknown-transport rejections unreachable. Replace test_update_weight_delta_rejects_colocate and test_update_weight_delta_rejects_unknown_transport (whose ValueError paths no longer fire) with a single test_update_weight_delta_disabled that asserts the guard raises for any delta config. Breadcrumb left to restore the per-condition tests when delta is verified and the guard is lifted. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * fix(sync): restore dropped weight-sync metrics chain + delta dispatch (slime parity) A mirror cross-check of the delta-weight-sync surface (slime #1806/#1991) found the #2014..#2125 sync had silently dropped several slime-faithful pieces: * extra_metrics logging chain — slime threads weight-update metrics from the actor through log_perf_data -> log_perf_data_raw. vime dropped the param at all three layers, so weight-update metrics were never logged. Restored: train_metric_utils.log_perf_data_raw(extra_metrics=...), data.log_perf_data passthrough, and actor passing self.weight_updater.pop_metrics(). * pop_metrics on UpdateWeightFromDistributed — the default (non-colocate, nccl) weight_updater. slime gives all three updaters a pop_metrics() stub so the actor can call it uniformly; vime kept it on tensor/disk but dropped it on distributed, which would AttributeError once the actor calls it. Restored the ~5-line stub (delta-specific plumbing stays dropped — vime+vLLM has no DeltaSpec). * actor delta-mode dispatch branch — restores the elif selecting UpdateWeightFromDistributedDelta. Dead code behind the validation guard that rejects --update-weight-mode=delta, so vime mirrors slime with the guard as the single divergence. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * chore(sync): align comments to the mechanical mirror Comment-only pass; no behavior change. Aligns vime comments to what a faithful slime->vime translation would carry: * Strip vime-divergence rationale markers (delta guard, opd-teacher-model, top-p fallback, colocate/delta tests). The rationale belongs in the divergence manifest, not inline; the guarded code + self-explanatory NotImplementedError messages stand on their own. * De-verbose vLLM-specific comments to mirror scale: the router-args block, the AsyncEngineArgs u FrontendArgs docstring, and the MoE-replay / streaming-rollout blocks that slime does not carry at that length. * Restore slime-original comments the sync had dropped or naively translated, with judgment translation of sglang-specific terms: session_id routing ("vLLM router", not the mechanical "Model Gateway"), "Prepare payload for vLLM server", the unique-session_id loop, and the pending-tasks wait. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> * docs(delta): note delta weight-sync not yet verified on vime+vLLM (PR vllm-project#286 review) Per review on PR vllm-project#286: delta weight sync is documented here but the arg guard disables --update-weight-mode=delta. Add a top-of-page note (en + zh) so users see it before hitting NotImplementedError at argparse. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> --------- Signed-off-by: aoshen02 <aoshen@inferact.ai> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
inferactinc/public:vime-*references withvllm/vime:*across CI, docs, and the release justfilevllm/vime:latest(rolling),vllm/vime:<version>(immutable),vllm/vime:test-latest(CI validation image)Files changed
.buildkite/gpu_suites.pyCI_IMAGE→vllm/vime:latestdocker/justfileIMAGE := "vllm/vime", drop redundantvime-tag prefixdocker/README.mddocs/en/+docs/zh/(5 files)Registry
vllm/vime:latestis live on DockerHub as a multi-arch manifest:linux/amd64sha256:c5b402c7…linux/arm64sha256:48c13127…