Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions docs/cookbook/diffusion/MiniMax/MiniMax-H3.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -667,6 +667,23 @@ An image condition here is semantic reference material rather than a
pixel-aligned first frame. Use the FL2VA tab when animating a screenshot from
that exact starting composition.

Set `reference_resize_mode` on the request to choose how reference images are
resized before encoding:

- `diffusers` (default) scales the short edge to 2048 pixels, including upscaling
small inputs. This preserves the original preprocessing behavior.
- `max` only downsizes references whose short edge exceeds 2048 pixels.
- `match` downsizes to the target canvas pixel area when needed, without
upscaling smaller references. Use this with the
[ModelTC Ref2VA Turbo LoRA](https://github.com/ModelTC/Minimax-H3-Turbo#note-on-reference-image-resizing),
whose training uses this policy.

All modes scale without cropping, then round each dimension to the nearest
multiple of 32. This rounding can slightly change the aspect ratio. The option
affects Ref2VA image references, not FL2VA keyframes, video, or audio references.
Changing it changes the conditioning input, so keep it fixed for quality and
performance comparisons; the server does not infer it from the adapter name.

```bash Command
curl -sS -X POST http://127.0.0.1:30010/v1/videos \
-H "Content-Type: application/json" \
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/sglang-diffusion/api/cli.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ Use `sglang generate --help` and `sglang serve --help` for the full argument lis
- `--lora-weight-name {FILE}`: select one adapter file from a repository that contains multiple LoRA revisions. The Hub download is filtered to that file plus JSON metadata, so unused weights are not downloaded.
- `--lora-alpha {N}`: supply the training alpha when a single-file adapter omits both per-layer alpha tensors and `adapter_config.json`. Do not set it when the adapter already records alpha metadata.
- PEFT `adapter_config.json` semantics are applied automatically for named adapter slots, RSLoRA, and per-layer `alpha_pattern`. Adapters that require unsupported auxiliary parameters or runtime behavior, such as DoRA, fail before weight injection instead of silently using ordinary LoRA math.
- `--lora-merge-mode {auto|merge|dynamic}`: choose how LoRA is applied. `auto` statically merges regular weights and uses dynamic LoRA for FSDP-sharded weights to avoid full-gather peaks.
- `--lora-merge-mode {auto|merge|dynamic}`: choose how LoRA is applied. `auto` normally merges regular weights, but keeps FSDP-sharded weights dynamic to avoid full-gather peaks. For a single adapter, it also stays dynamic when an estimate shows that merging would round away more than 50% of the update's norm. `merge` forces merging; `dynamic` applies the adapter at forward time.
- `--num-gpus {N}`: number of GPUs to use
- `--numa-node {N} [N ...]`: NUMA node for each GPU worker, the i-th value for the worker on local GPU i. When unset, each worker binds its CPUs and host memory to its GPU's NUMA node on multi-socket hosts, as srt does (CPUs only without `CAP_SYS_NICE`); set `SGLANG_AUTO_NUMA_BIND=0` to turn that off. Unbound workers on a two-socket host (e.g. GB200/GB300) can drift to the other socket, which slows host-bound steps and makes latency vary run to run.
- `--performance-mode {manual|auto|speed|memory}` / `--mode`: preset for latency/throughput and memory defaults. `auto` is the default and dispatches residency from selected-GPU headroom and workload type: image DiTs stay resident above the 45 GiB threshold, while video DiT placement remains model-specific. It uses FSDP only for validated DiT-offload replacement paths. `speed` keeps `torch.compile` disabled unless a model-specific deployment config opts in after validation; pass `--enable-torch-compile true` to enable it explicitly. Use `manual` to keep performance-related server args under explicit user control. Explicit offload, FSDP, and parallelism flags take precedence in all modes.
Expand Down
26 changes: 26 additions & 0 deletions docs/docs/sglang-diffusion/profiling.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,32 @@ sglang generate \
**Parameters:**
- `--profile-all-stages`: Used with `--profile`, profile all pipeline stages instead of just denoising

### Image API requests

For an already running image server, enable profiling on an individual request.
The JSON generations endpoint accepts the same controls with underscores:

```bash
curl -sS http://127.0.0.1:30000/v1/images/generations \
-H 'Content-Type: application/json' \
-d '{
"prompt": "A mountain lake at sunrise",
"seed": 0,
"profile": true,
"num_profiled_timesteps": 5,
"profile_all_stages": false
}'
```

For `/v1/images/edits`, pass these controls as multipart form fields alongside
the image and prompt: `-F profile=true -F num_profiled_timesteps=5`.
Add `-F profile_all_stages=true` to profile the full pipeline instead.

Traces are written on the server, not returned as image-response attachments.
Use `perf_dump_path` for a lightweight timing report instead of a profiler trace.
Run latency benchmarks separately with profiling disabled, since collecting a
trace adds overhead.

### Output Location

By default, trace files are saved in the ./logs/ directory.
Expand Down
Loading