From 2fc70ba0e644e6403eedd8220df0c431fdede926 Mon Sep 17 00:00:00 2001 From: Mick Qian Date: Thu, 15 Oct 2026 10:16:12 +0800 Subject: [PATCH] [diffusion] Document reference resizing, automatic LoRA merging and image profiling --- .../cookbook/diffusion/MiniMax/MiniMax-H3.mdx | 17 ++++++++++++ docs/docs/sglang-diffusion/api/cli.mdx | 2 +- docs/docs/sglang-diffusion/profiling.mdx | 26 +++++++++++++++++++ 3 files changed, 44 insertions(+), 1 deletion(-) diff --git a/docs/cookbook/diffusion/MiniMax/MiniMax-H3.mdx b/docs/cookbook/diffusion/MiniMax/MiniMax-H3.mdx index 025ba9a6f94c..c752a83d2ffb 100644 --- a/docs/cookbook/diffusion/MiniMax/MiniMax-H3.mdx +++ b/docs/cookbook/diffusion/MiniMax/MiniMax-H3.mdx @@ -667,6 +667,23 @@ An image condition here is semantic reference material rather than a pixel-aligned first frame. Use the FL2VA tab when animating a screenshot from that exact starting composition. +Set `reference_resize_mode` on the request to choose how reference images are +resized before encoding: + +- `diffusers` (default) scales the short edge to 2048 pixels, including upscaling + small inputs. This preserves the original preprocessing behavior. +- `max` only downsizes references whose short edge exceeds 2048 pixels. +- `match` downsizes to the target canvas pixel area when needed, without + upscaling smaller references. Use this with the + [ModelTC Ref2VA Turbo LoRA](https://github.com/ModelTC/Minimax-H3-Turbo#note-on-reference-image-resizing), + whose training uses this policy. + +All modes scale without cropping, then round each dimension to the nearest +multiple of 32. This rounding can slightly change the aspect ratio. The option +affects Ref2VA image references, not FL2VA keyframes, video, or audio references. +Changing it changes the conditioning input, so keep it fixed for quality and +performance comparisons; the server does not infer it from the adapter name. + ```bash Command curl -sS -X POST http://127.0.0.1:30010/v1/videos \ -H "Content-Type: application/json" \ diff --git a/docs/docs/sglang-diffusion/api/cli.mdx b/docs/docs/sglang-diffusion/api/cli.mdx index 2609532a34d0..0194d09eca40 100644 --- a/docs/docs/sglang-diffusion/api/cli.mdx +++ b/docs/docs/sglang-diffusion/api/cli.mdx @@ -86,7 +86,7 @@ Use `sglang generate --help` and `sglang serve --help` for the full argument lis - `--lora-weight-name {FILE}`: select one adapter file from a repository that contains multiple LoRA revisions. The Hub download is filtered to that file plus JSON metadata, so unused weights are not downloaded. - `--lora-alpha {N}`: supply the training alpha when a single-file adapter omits both per-layer alpha tensors and `adapter_config.json`. Do not set it when the adapter already records alpha metadata. - PEFT `adapter_config.json` semantics are applied automatically for named adapter slots, RSLoRA, and per-layer `alpha_pattern`. Adapters that require unsupported auxiliary parameters or runtime behavior, such as DoRA, fail before weight injection instead of silently using ordinary LoRA math. -- `--lora-merge-mode {auto|merge|dynamic}`: choose how LoRA is applied. `auto` statically merges regular weights and uses dynamic LoRA for FSDP-sharded weights to avoid full-gather peaks. +- `--lora-merge-mode {auto|merge|dynamic}`: choose how LoRA is applied. `auto` normally merges regular weights, but keeps FSDP-sharded weights dynamic to avoid full-gather peaks. For a single adapter, it also stays dynamic when an estimate shows that merging would round away more than 50% of the update's norm. `merge` forces merging; `dynamic` applies the adapter at forward time. - `--num-gpus {N}`: number of GPUs to use - `--numa-node {N} [N ...]`: NUMA node for each GPU worker, the i-th value for the worker on local GPU i. When unset, each worker binds its CPUs and host memory to its GPU's NUMA node on multi-socket hosts, as srt does (CPUs only without `CAP_SYS_NICE`); set `SGLANG_AUTO_NUMA_BIND=0` to turn that off. Unbound workers on a two-socket host (e.g. GB200/GB300) can drift to the other socket, which slows host-bound steps and makes latency vary run to run. - `--performance-mode {manual|auto|speed|memory}` / `--mode`: preset for latency/throughput and memory defaults. `auto` is the default and dispatches residency from selected-GPU headroom and workload type: image DiTs stay resident above the 45 GiB threshold, while video DiT placement remains model-specific. It uses FSDP only for validated DiT-offload replacement paths. `speed` keeps `torch.compile` disabled unless a model-specific deployment config opts in after validation; pass `--enable-torch-compile true` to enable it explicitly. Use `manual` to keep performance-related server args under explicit user control. Explicit offload, FSDP, and parallelism flags take precedence in all modes. diff --git a/docs/docs/sglang-diffusion/profiling.mdx b/docs/docs/sglang-diffusion/profiling.mdx index 71f34633ab41..3c4ad6bcc7ce 100644 --- a/docs/docs/sglang-diffusion/profiling.mdx +++ b/docs/docs/sglang-diffusion/profiling.mdx @@ -43,6 +43,32 @@ sglang generate \ **Parameters:** - `--profile-all-stages`: Used with `--profile`, profile all pipeline stages instead of just denoising +### Image API requests + +For an already running image server, enable profiling on an individual request. +The JSON generations endpoint accepts the same controls with underscores: + +```bash +curl -sS http://127.0.0.1:30000/v1/images/generations \ + -H 'Content-Type: application/json' \ + -d '{ + "prompt": "A mountain lake at sunrise", + "seed": 0, + "profile": true, + "num_profiled_timesteps": 5, + "profile_all_stages": false + }' +``` + +For `/v1/images/edits`, pass these controls as multipart form fields alongside +the image and prompt: `-F profile=true -F num_profiled_timesteps=5`. +Add `-F profile_all_stages=true` to profile the full pipeline instead. + +Traces are written on the server, not returned as image-response attachments. +Use `perf_dump_path` for a lightweight timing report instead of a profiler trace. +Run latency benchmarks separately with profiling disabled, since collecting a +trace adds overhead. + ### Output Location By default, trace files are saved in the ./logs/ directory.