[diffusion] quant: Add flag for runtime quantization

[avjves]

Motivation

Currently there's no way to quantize the model weights (linear layers) during runtime. Instead, only pre-quantized transformers are supported. This PR adds a new flag --quantization that can be used to do quantization on-the-fly.

Currently it only supports FP8 quantization, as out of the currently supported techniques, FP8 is the only one suited for runtime quantization.

Modifications

1. Adds a new flag --quantization for quantizing the model during runtime.
2. Fixes the quantization config fetching so that NVFP4 configs don't take precedence over CLI arguments.

Accuracy Tests

No quantization (bf16):

Offline quantization before running (fp8):

Runtime quantization (fp8):

---

This PR doesn't add any new quantization techniques, but the perf report below highlights that the quantization is used.

1. High-level Summary

Metric	Baseline	New	Diff	Status
E2E Latency	6482.07 ms	4994.77 ms	-1487.30 ms (-22.9%)	✅
Throughput	0.15 req/s	0.20 req/s	-	-

2. Stage Breakdown

Stage Name	Baseline (ms)	New (ms)	Diff (ms)	Diff (%)	Status
InputValidationStage	15.14	15.29	+0.15	+1.0%	⚪️
TextEncodingStage	17.76	17.78	+0.02	+0.1%	⚪️
ImageVAEEncodingStage	131.87	132.05	+0.18	+0.1%	⚪️
LatentPreparationStage	0.17	0.21	+0.04	+21.6%	⚪️
TimestepPreparationStage	0.27	0.29	+0.02	+6.7%	⚪️
DenoisingStage	6290.57	4796.93	-1493.65	-23.7%	🟢
DecodingStage	5.24	7.71	+2.47	+47.1%	⚪️

Speed Tests and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

[gemini-code-assist]

Code Review

This pull request introduces runtime FP8 quantization support by adding a --quantization CLI argument and updating the quantization configuration resolution process. Feedback indicates that the refactoring of _resolve_quant_config contains regressions, including the use of stale configuration names and incorrect early returns that skip essential NVFP4 inference and metadata probing.

[mickqian]

/tag-and-rerun-ci

[mickqian]

could you update the doc as a follow up

[avjves]

@mickqian I updated the docs now :)

[mickqian]

/rerun-failed-ci

[avjves]

Closing, as #20922 was merged that already adds this support.