Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@ SGLang supports **mix-bits** quantization (independently defines and loads each
<tr>
<th>Quantization scheme</th>
<th>Layer type</th>
<th>Ascend A2 Series Products Supported</th>
<th>Ascend A3 Series Products Supported</th>
<th>Ascend 950PR/DT Series Products Supported</th>
<th>A2 Series Supported</th>
<th>A3 Series Supported</th>
<th>950PR/DT Series Supported</th>
<th>Diffusion models</th>
</tr>
</thead>
Expand Down Expand Up @@ -133,9 +133,9 @@ SGLang supports **mix-bits** quantization (independently defines and loads each
<tr>
<th>Quantization scheme</th>
<th>Layer type</th>
<th>Ascend A2 Series Products Supported</th>
<th>Ascend A3 Series Products Supported</th>
<th>Ascend 950PR/DT Series Products Supported</th>
<th>A2 Series Supported</th>
<th>A3 Series Supported</th>
<th>950PR/DT Series Supported</th>
</tr>
</thead>
<tbody>
Expand Down Expand Up @@ -169,9 +169,9 @@ GPTQ on Ascend support
<tr>
<th>Quantization scheme</th>
<th>Layer type</th>
<th>Ascend A2 Series Products Supported</th>
<th>Ascend A3 Series Products Supported</th>
<th>Ascend 950PR/DT Series Products Supported</th>
<th>A2 Series Supported</th>
<th>A3 Series Supported</th>
<th>950PR/DT Series Supported</th>
</tr>
</thead>
<tbody>
Expand Down Expand Up @@ -212,9 +212,9 @@ GPTQ on Ascend support
<tr>
<th>Quantization scheme</th>
<th>Layer type</th>
<th>Ascend A2 Series Products Supported</th>
<th>Ascend A3 Series Products Supported</th>
<th>Ascend 950PR/DT Series Products Supported</th>
<th>A2 Series Supported</th>
<th>A3 Series Supported</th>
<th>950PR/DT Series Supported</th>
</tr>
</thead>
<tbody>
Expand Down Expand Up @@ -255,9 +255,9 @@ Compressed-tensors (LLM Compressor) on Ascend support:
<tr>
<th>Quantization scheme</th>
<th>Layer type</th>
<th>Ascend A2 Series Products Supported</th>
<th>Ascend A3 Series Products Supported</th>
<th>Ascend 950PR/DT Series Products Supported</th>
<th>A2 Series Supported</th>
<th>A3 Series Supported</th>
<th>950PR/DT Series Supported</th>
</tr>
</thead>
<tbody>
Expand Down Expand Up @@ -298,9 +298,9 @@ Compressed-tensors (LLM Compressor) on Ascend support:
<tr>
<th>Quantization type</th>
<th>Layer type</th>
<th>Ascend A2 Series Products Supported</th>
<th>Ascend A3 Series Products Supported</th>
<th>Ascend 950PR/DT Series Products Supported</th>
<th>A2 Series Supported</th>
<th>A3 Series Supported</th>
<th>950PR/DT Series Supported</th>
</tr>
</thead>
<tbody>
Expand Down Expand Up @@ -504,8 +504,8 @@ SGLang-Diffusion supports MXFP8 online and offline quantization for diffusion mo
<th><code>quant_type</code> in JSON</th>
<th>Scheme class</th>
<th>Mode</th>
<th>Ascend A2/A3 Series Products Supported</th>
<th>Ascend 950PR/DT Series Products Supported</th>
<th>A2/A3 Series Supported</th>
<th>950PR/DT Series Supported</th>
<th>Trigger</th>
</tr>
</thead>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2100,12 +2100,6 @@ If the value is int8, you must also set the environment variable:DEEP_NORMAL_MOD
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>bool flag<br/> (set to enable)</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>A2/A3 Series</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`--cuda-graph-bs-decode`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`None`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>List[int]</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>A2/A3 Series</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`--disable-cuda-graph`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`False`</td>
Expand Down Expand Up @@ -2244,30 +2238,12 @@ If the value is int8, you must also set the environment variable:DEEP_NORMAL_MOD
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>bool flag<br/> (set to enable)</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>A2/A3 Series</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`--cuda-graph-bs-prefill`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`None`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>Type: JSON<br/> list</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>A2/A3 Series</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`--cuda-graph-tc-compiler`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`eager`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>`eager`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>A2/A3 Series</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`--torch-compile-max-bs`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`32`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>Type: int</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>A2/A3 Series</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`--cuda-graph-max-bs-prefill`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`None`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>Type: int</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>A2/A3 Series</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`--enable-p2p-check`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`False`</td>
Expand Down
Loading