[CI] Shard Humming H100 eval

[khluu]

Why

lm-eval-humming-f16-h100 took 62.6 minutes in full CI build 83851. Its 14 model configurations ran serially even though each evaluation is independent.

This does not duplicate an open pull request. Open PRs were searched by the exact step key and Humming/H100 sharding terms before implementation.

What changed

• run the step with three Buildkite jobs
• partition all 14 configurations into three explicit H100 lists using observed build 83851 per-config durations
• keep each configuration in exactly one shard

The measured baseline loads are 20.52, 20.38, and 20.74 minutes of test work, leaving headroom below the 30-minute target.

Validation

• baseline model evaluation: 14/14 configurations passed in build 83851
• config partition check: 14/14 unique entries, no omissions or duplicates, all referenced YAML files exist
• selected-step pipeline generation: three parallel H100 jobs with the image-build dependency
• targeted build 83895: all three shards passed, with exact 14/14 configuration coverage and no omissions or duplicates
• targeted shard wall times: 20.79, 20.33, and 19.33 minutes (1.48-minute max/min spread; 1.08x imbalance)
• outer job/bootstrap + teardown overhead around pytest: 0.83, 0.80, and 0.74 minutes per shard
• critical path: 20.79 minutes versus 62.63 minutes in build 83851 (3.01x faster)
• observed total GPU time: 60.45 versus 62.63 GPU-minutes in the single baseline run; the -2.18 minute delta is normal run-to-run variance, not negative sharding overhead
• uvx pre-commit run --files ...: passed
• git diff --check: passed

AI assistance was used. The submitter will review and validate the change before marking it ready.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.