[Feat] ShareGPT input len

[jiminpark-moreh]

Purpose

Add ShareGPT input len option.

The ShareGPT dataset uses pre-built questions as prompts, so the input length varies for each request. Since we need performance with uniform input sequence length, we added an input length option to ShareGPT. When --sharegpt-input-len is specified, new input is created by concatenating existing questions into a single prompt.

Test Plan

vllm bench serve \
  --backend vllm \
  --model "deepseek-ai/DeepSeek-R1" \
  --metric-percentiles "90" \
  --percentile-metrics "itl,tps,ttft,e2el" \
  --host "mif-istio.cluster.svc.cluster.local" \
  --port 80 \
  --num-prompts 32400 \
  --max-concurrency 10800 \
  --request-rate 78 \
  --ignore-eos \
  --ready-check-timeout-sec 0 \
  --max-connections-per-worker 1296 \
  --dataset-name sharegpt \
  --dataset-path /app/dataset/ShareGPT_V3_unfiltered_cleaned_split.json \
  --sharegpt-input-len 1000 \
  --sharegpt-output-len 1000

Test Result

1. Creating new prompts on benchmark

[3240/32400] new prompts are created
[6480/32400] new prompts are created
[9720/32400] new prompts are created
INFO 12-08 06:51:23 [datasets.py:245] Oversampled requests to reach 32400 total samples.
...

2. On benchmark result, "Total input tokens" will be the number of num-prompts x sharegpt-input-len

============ Serving Benchmark Result ============
Number of worker processes:              25
Successful requests:                     32400
Maximum request concurrency:             10800
Request rate configured (RPS):           80.00
Benchmark duration (s):                  636.88
Total input tokens:                      32400000
Total generated tokens:                  32400000
...

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.