feat: parse diffusion worker args natively from sglang's diffusion ServerArgs

[marckarp]

Overview

Stacked on #13343 — do not merge first. This PR's base is that branch; GitHub will retarget to main when it merges. Design direction (build on the native diffusion parser, on top of the 0.5.17 bump) was agreed in team review.

Image/video diffusion workers currently parse CLI args with the LLM ServerArgs parser, hand-copy 13 fields into a SimpleNamespace stub, and call DiffGenerator.from_pretrained() with 5 hardcoded kwargs. Consequences:

• Nearly every diffusion engine option (torch compile, quantization, batching, warmup, attention backend, …) is silently stuck at its default.
• Unknown or LLM-only flags are silently absorbed — a typo'd flag and a working flag are indistinguishable.
• --enable-torch-compile collides with the same-named LLM flag: it parses into the LLM ServerArgs and never reaches the diffusion config; passing it a value crashes the worker.
• The stub itself drops flags it claims to carry: --trust-remote-code never reached the engine, and dist_timeout was forced to None, disabling the engine's default distributed-init timeout.

What this PR does

Build the worker CLI from sglang's native diffusion ServerArgs, mirroring how the LLM path builds its CLI from the LLM ServerArgs:

• diffusion_args.py (new) — native parser + DiffusionWorkerArgs adapter. Engine fields delegate to the natively parsed diffusion ServerArgs; Dynamo-side settings (--served-model-name, --enable-metrics) are registered only when the engine parser does not define them natively, so when sglang ships its own (already on sglang main) it wins automatically. Fields Dynamo's shared worker code probes on any server_args are pinned to inert values. Explicitly-provided flags are passed through the parser's _sglang_explicit_arg_names side channel so resolution never depends on process argv (keeps the module testable and embeddable). Version-tolerant import for the server_args module/package layout change.
• args.py — diffusion/video workers branch to the native parser before any LLM-specific processing; the stub is deleted.
• init_diffusion.py — both image and video inits hand the full parsed config to DiffGenerator.from_server_args(), replacing the 5 hardcoded kwargs.
• Tests — 9 hermetic unit tests (fake engine parser injected via sys.modules; no GPU, no model).

Breaking change / migration

LLM-only flags on diffusion worker command lines — previously silently ignored — now fail at startup with unrecognized arguments. Example: a deployment passing --skip-tokenizer-init to an --image-diffusion-worker must drop that flag. This is the intended contract (same as the LLM path): loud rejection is what makes the newly reachable flags trustworthy. If reviewers prefer a softer landing (warn-and-ignore for one release), that's a small follow-up.

Verification

• Parser against real sglang 0.5.17 (upstream lmsysorg/sglang:v0.5.17-cu130-runtime): 126 engine fields reachable (previously 5); invalid values rejected with actionable messages (--performance-mode: invalid choice: ... choose from 'manual', 'auto', 'speed', 'memory'); unknown flags rejected loudly; Dynamo-side flags kept out of the engine config.
• Unit tests: 9/9.
• Live image e2e (FLUX.1-schnell, 1×B200): worker boots through the native path; --warmup-mode server — previously unreachable — confirmed active in the resolved engine config and engine logs; /v1/images/generations returns HTTP 200; warm request 0.97 s at 1024².
• Live video e2e (Wan2.1-T2V-1.3B, 1×B200): worker boots through the native path with --video-generation-worker; full pipeline runs and /v1/videos returns a playable file.
• Resolved-config diff, stub vs native (107 fields): zero unintended changes. The only diffs were the explicitly passed warmup flags, the two stub bugs now fixed (trust_remote_code, dist_timeout), and a per-boot random port.
• Assembled check: sglang v0.5.17 image + current dynamo wheel + this branch's files boots a frontend+worker pair and serves image generation end-to-end (HTTP 200). The official container build of this combination lands via CI — which also covers the image_diffusion.sh launch-script run that chore(sglang): bump to 0.5.17 #13343's validation had to skip for lack of model access.

Notes for reviewers

• The old stub set dllm_algorithm=False, which made diffusion_worker = server_args.dllm_algorithm is not None accidentally True for image/video workers; it was never load-bearing because main.py dispatches image/video workers first. The new path sets it False explicitly (dllm = text-diffusion LLMs, unrelated to image/video generation).
• num_gpus derivation (tp_size * dp_size) moves from Dynamo code into sglang's own ServerArgs resolution, and --num-gpus is now directly settable like any other native flag.
• Part of the same fix series as fix: honor the OpenAI n parameter in the SGLang image diffusion handler #12970 and fix: surface image generation errors instead of returning empty HTTP 200 #13000.

Part of DYN-3974

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[jain-ria]

Correctly rejects unsupported arguments, but both checked-in diffusion launchers still pass in the LLM-only --skip-tokenizer-init flag (image, video). Can we update + double check both scripts?

[marckarp]

Fixed in #13667: --skip-tokenizer-init removed from both launch scripts, and the video launcher now uses the canonical --tp-size. Also swept the repo for other callers passing LLM-only flags to diffusion workers (docs, tests, configs) — these two scripts were the only ones. Both scripts were executed end to end against sglang 0.5.17 and serve generation requests; the old argument vector is rejected with unrecognized arguments: --skip-tokenizer-init.

[jain-ria]

parse_known_args() can accept an abbreviated or aliased option and store it under a different name, but this records the raw spelling, so then SGLang then may not recognize the corresponding field as explicitly provided. For example, the existing video launcher’ passes --tp 2, accepting it as an abbreviation for --tp-size, but this code records the raw name tp. Because SGLang does not map tp back to tp_size, it treats tp_size as unspecified and falls back to 1.

[marckarp]

Great catch — fixed in #13667. The side channel now resolves each provided flag to its parser destination (exact match or unique prefix, mirroring argparse's own abbreviation rules) instead of recording the raw spelling. Unit-tested: --tp 2 lands as tp_size=2 in both the parsed config and the explicit-args set. Flags not registered on the parser (e.g. dynamic --<component>-path) still fall back to the normalized raw name so the engine's own resolution handles them.

[jain-ria]

This update removed the previous num_gpus = tp_size * dp_size calculation, defaulting to 1. Any call that runs with TP or DP > 1 without also passing --num-gpus will have an inconsistent configuration.

[marckarp]

Confirmed and fixed in #13667 — the engine defaults num_gpus to 1 and does not derive it from parallelism degrees, so this was a real regression. Restored num_gpus = tp_size * dp_size when --num-gpus is not explicitly provided (an explicit --num-gpus wins). Covered by two unit tests.

[marckarp]

GitHub does not allow reopening this PR after the base-branch deletion, so it continues in #13667 — the same change rebased onto main, with all three review comments addressed there (each has a threaded reply above with the fix and test evidence). Thanks @jain-ria — all three were real issues.