[benchmark] Port benchmark request sent optimization to benchmark_serving

[Jialin]

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Purpose

Port changes in #21108 to benchmark_serving script as well. However, instead of copypasta the code changes, we directly let benchmark_serving scripts depends on serve.py functions to avoid code duplication.

In a long run, we would completely get rid of benchmark_serving (#21206). However, that's out of the scope of this PR which is focusing on closing gaps among these 2 benchmark scripts.

Test Plan

Check request throughput in the following scenarios:

• fix request rate
• unlimited request rate

Test Result

Fixed request rate benchmark run

• Request Rate 200: Throughput improved 184.01 -> 199.89
• Request Rate 1000: Throughput improved 687.95 -> 771.87

Before (Request Rate 200)

After (Request Rate 200)

Before (Request Rate 1000)

After (Request Rate 1000)

python benchmarks/benchmark_serving.py \
    --dataset-name random \
    --model facebook/opt-125m \
    --served-model-name facebook/opt-125m \
    --random-input-len 700 \
    --random-output-len 1 \
    --endpoint /v1/completions \
    --ignore-eos \
    --host localhost \
    --port 8000 \
    --request-rate 200 \
    --num-prompts 10000

Unlimited request rate

Request Rate slight improvements: 418.24 -> 433.88

Additional observations It UNEXPECTED that the server request throughput of Unlimited request rate is worse request rate 1000, which indicates there're weakness of the benchmark scripts or server for churn of requests. We're going to follow up for the investigation as a separate issue.

Before

After

(Optional) Documentation Update

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[gemini-code-assist]

Code Review

This pull request refactors benchmarks/benchmark_serving.py to reuse the request generation logic from vllm/benchmarks/serve.py, effectively reducing code duplication. The change also includes a micro-optimization in vllm/benchmarks/serve.py to avoid unnecessary time.time() calls when the request rate is unlimited. The changes are well-reasoned and the provided test results demonstrate a positive performance impact. The code appears correct and improves maintainability.

[yeqcharlotte]

awesome thanks!

[houseroad]

Looks good, thanks!