[Thangnguyenvn98]

Overview

Enrolls the remaining LM Eval jobs in automatic_shard: 30 jobs (16 steps and 14 AMD mirrors). Part of a series that enrolls every remaining test step, split by test area for review. No commands change.

Claims

Step	Plan	AMD mirror
kv-offload-large	2 shards	1 shard
gpqa-eval-gpt-oss-2xb200	1 shard	1 shard
lm-eval-turboquant-k8v4	1 shard	1 shard
kv-offload-small	1 shard	1 shard
lm-eval-humming-h100-deepep	1 shard	none
kv-offload-medium	1 shard	1 shard
lm-eval-dspark-watermark-2xh100	1 shard	1 shard
lm-eval-nemotron-3-super-8xh200	1 shard	1 shard
gpqa-eval-gpt-oss-spark	1 shard	1 shard
mrcr-eval-small-models	1 shard	1 shard
lm-eval-watermarking	1 shard	1 shard
amd-lm-eval-large-models-rocm-harness	1 shard	none
amd-lm-eval-large-models	1 shard	none
amd-lm-eval-large-models-harness	1 shard	none
amd-lm-eval-octave-mi300	1 shard	none
amd-lm-eval-small-models-harness	1 shard	none
lm-eval-small-models	already enrolled	1 shard
moe-refactor-integration-test-h100-temporary	already enrolled	1 shard
moe-refactor-integration-test-b200-temporary	already enrolled	1 shard
moe-refactor-integration-test-b200-dp-temporary	already enrolled	1 shard

• One-shard and one-job plans run the same tests as on main, as one job, and split by themselves once their tests grow.
• Plans of 2 or more shards cost one extra job setup per shard and bring each shard under the 20-minute target.

Validation

• Planned each step and mirror with ci-infra main's planner against main's timings; every plan is within 10% of the same plan run as one job.
• The sharding harness (each shard's commands run against a mock test tree, as a PR build and as main with the OTel wrapper) shows every test runs exactly once wherever a plan has 2 or more shards.
• Every step and mirror passes on recent main runs, nightly included.
• /ci run (98455) plus a scoped build (98466) for the optional steps and AMD mirrors on this head: every enrolled job passed, and every shard finished under 22 minutes.

Details

• Not plain pytest steps (shell scripts, torchrun, examples) are enrolled too; the generator keeps them as one job. Standalone AMD steps plan like any other since Fix automatic sharding for standalone AMD jobs and selected tests ci-infra#871.
• caplog. Files that assert on logs sit in one-shard plans here, so their order in the process doesn't change.
• KV-Offload Large splits its one file by test into 2 shards: the wait drops from about 30 to 18 minutes, for about 16% more machine time because each shard loads its model. Kept for the shorter wait, as with Multimodal Extended Pooling.
• Timeouts unchanged. timeout_in_minutes stays per job. Fallback-aware timeouts belong in a central timeout setting.

🤖 Generated with Claude Code

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing (anything written below this line will be removed by GitHub Actions)

[Thangnguyenvn98]

/ci run

[github-actions]

✅ Triggered Buildkite CI #98455 for commit 2ac08cf4d336.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.