[Bug Fix] Reject incompatible combination of --disable-cuda-graph-padding and --enable-torch-compile

[ppraneth]

Motivation

Fixes #23865.

When a user passes both --disable-cuda-graph-padding and --enable-torch-compile together, the engine never finishes initializing. It silently hangs for many minutes (or indefinitely on larger models) with no indication that anything is wrong.

The root cause is a batch size explosion in the CUDA/CPU graph capture step. With padding enabled, the capture list is a small fixed set of bucket sizes like [1, 2, 4, 8, 12, 16, 24]. With padding disabled, _generate_cuda_graph_batch_sizes and _generate_cpu_graph_batch_sizes both expand this to every integer from 1 up to cuda_graph_max_bs (for example, 1 through 24 or up to 160 on small GPUs). Because CPUGraphRunner runs a full torch.compile and Triton AUTOTUNE cycle for each batch size, the number of kernel benchmarks grows from roughly 28 (7 buckets x 4 matmul shapes) to hundreds or thousands. In the reproduction on an RTX 4090, the process was killed after 3 minutes while still at batch size 11 out of 24.

The two flags are also semantically incompatible at runtime. --disable-cuda-graph-padding means the server will route exact incoming batch sizes without rounding up. If those exact sizes were not pre-compiled, torch.compile would trigger a new compilation on the fly during request serving, causing latency spikes for real traffic.

Modifications

• Added a validation check in ServerArgs.check_server_args() in python/sglang/srt/server_args.py that raises an AssertionError immediately at engine startup when both --disable-cuda-graph-padding and --enable-torch-compile are passed together.
• The error message explains the cause (O(max_batch_size) autotune explosion) and tells the user which flag to remove, so there is no silent hang and no confusion about what went wrong.
• No behavior change for any other flag combination.

Accuracy Tests

Speed Tests and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[ppraneth]

@ShangmingCai Can you please review this pr?

[ppraneth]

@b8zhong Can you review this pr?

[ShangmingCai]

Thx for the PR. I am not an expert on CUDA graphs and Torch compile. This PR appears to be a guard rather than a fix, but the use case explanation sounds reasonable, so I think we can merge this. If people are requiring the mixed use of these configs, but hitting this assertion error in the future, then they could propose a real fix and remove this assertion at that time.