Skip to content

[Bugfix][Rollout] Restore generation with vLLM 0.30 - #1

Draft
0z5a wants to merge 2 commits into
stack/vime-dev-h100-20260928from
fix/vllm-030-rollout-endpoints
Draft

0z5a wants to merge 2 commits into
stack/vime-dev-h100-20260928from
fix/vllm-030-rollout-endpoints

Conversation

@0z5a

@0z5a 0z5a commented Sep 28, 2026 •

Copy link
Copy Markdown
Owner

Incremental review: Files changed.

Base: stack/vime-dev-h100-20260928 at c8be49d2b08a17d66870f01ded72f2a8b59191f8.

Restore startup and rollout endpoints with vLLM 0.30. Move the three serving CLI imports from entrypoints.openai to entrypoints.launchers, align the existing import fixture and set enable_scale_out=True for /inference/v1/generate.

Validation Before Candidate
Serving CLI Import failure Passed
Generation after import-only fix HTTP 404 128 real requests completed on 2×H100
CLI/config/engine checks on H100 and H20 — 110 passed
Full Qwen3-4B run, 2×H20, TP=2 Startup blocked Rollout → Adam update → full checkpoint → publication → normal exit
H20 timing Candidate Speed ratio
Generation 163.184 s N/A: baseline cannot start
Actor training 88.415 s N/A
Reported step 252.854 s N/A
Whole process 768.372 s N/A

Qwen3-4B revision 1cfa9a7208912126459214e8b04321603b3df60c, 16 prompts × 8 completions, 4096-token cap. Reward was 16/128, truncation 112/128 and gradient norm 0.1077845. A second process restored model, optimizer, scheduler and dataset cursor and completed five further online updates with finite nonzero gradients, checkpoint, publication and normal exit. Its warm median step was 190.601 s with KV cache increased from 16 to 32 GiB per card; that is not a code speedup.

The existing import fixture remains required by the namespace change; the added endpoint test was removed. Resident-actor optimization is reviewed separately in #2.

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant