Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions docs/cookbook/diffusion/Cosmos/Cosmos3.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ curl -sS -X POST http://127.0.0.1:30010/v1/images/generations \
"guidance_scale": 6.0,
"flow_shift": 3.0,
"seed": 0,
"extra_args": {
"extra_body": {
"use_resolution_template": false,
"guardrails": true
}
Expand All @@ -141,7 +141,7 @@ curl -sS -X POST http://127.0.0.1:30010/v1/images/generations \
"n": 1,
"guidance_scale": 1.0,
"seed": 0,
"extra_args": {
"extra_body": {
"use_resolution_template": false,
"guardrails": true
}
Expand Down Expand Up @@ -390,7 +390,10 @@ Cosmos3 omnimodal fields are accepted as extra JSON fields or multipart form fie
- `action_view_point`: viewpoint used in the structured action caption.
- `action_normalization`: dataset normalization mode, such as `quantile`, `meanstd`, or `minmax`.

Put model-specific compatibility knobs in `extra_params` for video requests, or `extra_args` for image requests:
Pass model-specific controls through `extra_body` with the OpenAI Python SDK.
Raw JSON may keep them at the top level; multipart video requests should put
them in the `extra_params` JSON object. The legacy image `extra_args` container
remains accepted for compatibility, but new clients should use `extra_body`:

- `use_duration_template`: whether to append SGLang's generated duration suffix to video prompts.
- `use_resolution_template`: accepted for vLLM-Omni request compatibility.
Expand Down
25 changes: 25 additions & 0 deletions docs/cookbook/diffusion/LTX/LTX2.5.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,23 @@ The prediction is clamped to `--auto-duration-min-seconds` /
`--auto-duration-max-seconds` (default 1–20 s) and snapped to the VAE's temporal
grid, so the result is always a valid frame count. It overrides `--num-frames`.

For an online server, pass the same LTX-2.5-only controls through `extra_body`:

```python Python
from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:30010/v1")
video = client.videos.create(
model="Lightricks/LTX-2.5-Diffusers",
prompt="A red fox walking through a snowy forest at dawn.",
extra_body={
"auto_duration": True,
"auto_duration_min_seconds": 2.0,
"auto_duration_max_seconds": 8.0,
},
)
```

### 4.4 Two-stage (higher quality)

Stage 1 runs at half the requested resolution, the latents are upsampled 2x, and
Expand Down Expand Up @@ -290,6 +307,14 @@ sglang serve \
--load-diffusion-decoder
```

```python Python
video = client.videos.create(
model="Lightricks/LTX-2.5-Diffusers",
prompt="A red fox walking through a snowy forest at dawn.",
extra_body={"use_diffusion_decoder": True},
)
```

This keeps the default server footprint unchanged while still allowing VAE and
diffusion-decoder requests to share one server. When GPU memory is constrained,
`--cpu-offload-components diffusion_decoder` keeps the optional decoder on CPU
Expand Down
92 changes: 92 additions & 0 deletions docs/cookbook/diffusion/LongCat/LongCat-Image.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
---
title: LongCat-Image
metatags:
description: "Deploy LongCat-Image with SGLang Diffusion and its native in-process Qwen2.5-VL prompt rewriter."
---

import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';

<DiffusionModelTags tags={["image", "text-to-image", "prompt rewriting", "Qwen2.5-VL"]} />

## 1. Model Introduction

[LongCat-Image](https://huggingface.co/meituan-longcat/LongCat-Image) is a
text-to-image model from Meituan. SGLang runs its Qwen2.5-VL prompt rewriter
in process with the native SGLang runtime before text encoding and denoising.

The native pipeline keeps prompt rewriting and diffusion behind one OpenAI-compatible
image endpoint. Rewriting is enabled by default for stronger prompt expansion, but
each request can disable it when lower latency matters more than the rewritten prompt.

## 2. Installation

Install SGLang with the diffusion dependencies:

```bash Command
pip install -e "python[diffusion]"
```

For other installation options, see the
[SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).

## 3. Serve the model

```bash Command
sglang serve \
--model-path meituan-longcat/LongCat-Image \
--performance-mode auto \
--port 30010
```

Prompt rewriting is enabled by default for LongCat-Image. It adds an
autoregressive Qwen2.5-VL pass before diffusion; set
`enable_prompt_rewrite=false` on a request when lower latency is more important
than rewritten prompt quality.

## 4. Generate an image

```python Python
import base64
from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://127.0.0.1:30010/v1")

response = client.images.generate(
model="meituan-longcat/LongCat-Image",
prompt="A quiet bookshop on a rainy evening, warm light in the windows",
n=1,
response_format="b64_json",
)

image_bytes = base64.b64decode(response.data[0].b64_json)
with open("longcat_image.png", "wb") as f:
f.write(image_bytes)
```

To skip prompt rewriting with the OpenAI client, pass the model-specific request
field through `extra_body`:

```python Python
response = client.images.generate(
model="meituan-longcat/LongCat-Image",
prompt="A quiet bookshop on a rainy evening",
extra_body={"enable_prompt_rewrite": False},
)
```

## 5. Memory placement

Use the unified component-residency selector when the complete pipeline does
not fit on the accelerator. For example, keep the repeatedly used DiT resident
while moving auxiliary components to CPU between stages:

```bash Command
sglang serve \
--model-path meituan-longcat/LongCat-Image \
--component-residency dit=resident text_encoder=component-offload vae=component-offload \
--pin-cpu-memory \
--port 30010
```

See [Component Residency](/docs/sglang-diffusion/api/cli#component-residency)
for mode semantics and compatibility with the existing CPU-offload flags.
6 changes: 6 additions & 0 deletions docs/cookbook/diffusion/intro.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,12 @@ Image models generate one image request as a bounded denoising job, usually with
href="/cookbook/diffusion/Qwen-Image/Qwen-Image"
img="/cards/logos/qwen.png"
/>
<Card
title="LongCat-Image"
mode="card"
href="/cookbook/diffusion/LongCat/LongCat-Image"
img="/cards/logos/meituan.png"
/>
<Card
title="Z-Image"
mode="card"
Expand Down
7 changes: 7 additions & 0 deletions docs/docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -1497,6 +1497,13 @@
"cookbook/diffusion/Qwen-Image/Qwen-Image-Edit"
]
},
{
"group": "LongCat-Image",
"tag": "NEW",
"pages": [
"cookbook/diffusion/LongCat/LongCat-Image"
]
},
{
"group": "Z-Image",
"pages": [
Expand Down
23 changes: 23 additions & 0 deletions docs/docs/sglang-diffusion/api/openai_api.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,29 @@ curl -sS "http://localhost:30010/v1/models/wan-t2v"

## Endpoints

### Model-specific request fields

The image and video request schemas contain only OpenAI fields and stable
SGLang extensions shared across model families. Put model-specific controls in
`extra_body` when using the OpenAI Python client:

```python Python
from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:30010/v1")
response = client.images.generate(
model="ideogram-ai/ideogram-4-nf4",
prompt="A typeset poster",
extra_body={"preset": "V4_QUALITY_48"},
)
```

Raw JSON requests may send these fields at the top level. Multipart video
requests may send individual declared form fields or one JSON object in
`extra_params`. SGLang forwards model-specific extensions only when the active
model declares them, so a field for one model family does not silently change
another model.

### Image Generation

The server implements an OpenAI-compatible Images API under the `/v1/images` namespace.
Expand Down
2 changes: 1 addition & 1 deletion python/sglang/multimodal_gen/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ SGLang diffusion features an end-to-end unified pipeline for accelerating diffus
## Key Features

SGLang Diffusion has the following features:
- Broad model support: Wan, FastWan, FLUX, Qwen-Image, Z-Image, Ideogram 4, Krea-2, Cosmos3, LTX-2/LTX-2.3, MiniMax-H3, LingBot Video MoE, LingBot World, SANA-Video/SANA-WM, JoyEcho, MOVA, GLM-Image, ERNIE-Image, Hunyuan3D, and more
- Broad model support: Wan, FastWan, FLUX, Qwen-Image, LongCat-Image, Z-Image, Ideogram 4, Krea-2, Cosmos3, LTX-2/LTX-2.3/LTX-2.5, MiniMax-H3, LingBot Video MoE, LingBot World, SANA-Video/SANA-WM, JoyEcho, MOVA, GLM-Image, ERNIE-Image, Hunyuan3D, and more
- Fast inference speed: empowered by optimized `sgl-kernel` kernels, scheduler/runtime improvements, caching acceleration, and native diffusion hot-path optimizations
- Ease of use: OpenAI-compatible api, CLI, and python sdk support
- Multi-platform support:
Expand Down
Loading
Loading