refactor(runners): one launcher per pool for B200 Nscale

[cquil11]

Description

There is exactly one B200 Nscale runner pool (b200-nscale-slurm_00.._10, labels b200-nscale / cluster:b200-nscale), and the workflows dispatch with bash ./runners/launch_${RUNNER_NAME%%_*}.sh, so the only launcher that pool ever resolves to is runners/launch_b200-nscale-slurm.sh. runners/launch_b200-nscale-compat.sh (renamed from the retired pool's launch_b200-dgxc.sh in #2746) was only ever reached by an exec from that file. Two files for one pool made it look like there were two pools, and the b200-nscale-compat alias in runtime_settings.sh matched no runner.

This PR restores a 1:1 mapping between runner pools and launcher files for B200 Nscale and writes the rule down.

Changes

• runners/launch_b200-nscale-slurm.sh: absorbs the compat script. The three execution paths are selected once near the top and named in the log:
  • native-srt: the multi-node lanes whose srt-slurm recipes are maintained against this cluster (DSV4 / Kimi K2.6 / Kimi K3 / GLM-5.2 FP4, GLM-5.1 FP8 TileRT) — the old slurm launcher body.
  • multinode-srt: every other multi-node job — the old compat multi-node body.
  • single-node: salloc + srun of the benchmarks/single_node script — the old compat single-node body.
    Each body is carried over verbatim as a function. The only consolidated code is the enroot squash import, which the two files had duplicated: one import_squash / enroot_uri_for_image / ensure_writable_squash_dir now serves both srt-slurm paths using the caller's SQUASH_DIR and SQUASH_LOCK_TIMEOUT. SLURM_PARTITION / SLURM_ACCOUNT come from runtime_settings.sh via check_env_vars instead of being hardcoded a second time in the native path.
• runners/launch_b200-nscale-compat.sh: deleted.
• runners/runtime_settings.sh: drop the b200-nscale-compat case alias that no runner resolves to.
• benchmarks/single_node/agentic/glm5.2_fp8_b200_sglang_mtp.sh: comment now names the real launcher.
• AGENTS.md: new "Runner launchers (one file per pool)" section stating the routing contract and forbidding fallback launchers and launcher-name aliases.

Behavior preservation

• Path selection: the old two-file gate logic and the new uses_native_srt_lane were run side by side over 840 combinations of IS_MULTINODE × FRAMEWORK × MODEL_PREFIX/PRECISION × SPEC_DECODING × IS_AGENTIC with 0 mismatches (31 combinations select the native lane, as before).
• Bodies: diff of each function against its source region in the old files shows only the squash-helper consolidation, the check_env_vars moves to path entry, and one comment reword. The dead duplicated minimaxm2.5 squash-dir block and the unused NSCALE_MODEL_ROOT constant are gone.
• DCGM exporter: the shared URI helper yields the identical docker://nvcr.io#nvidia/k8s/dcgm-exporter:... reference and lock key that the old manual nvcr.io# substitution produced.
• bash -n and shellcheck -S warning are clean on both touched scripts (the old files had shellcheck findings).
• Not run on the cluster. The first full-sweep on B200 Nscale is the real validation; a single-node job, a multinode-srt job (for example DSR1 FP8 dynamo-sglang) and a native-srt job (DSV4 FP4 dynamo-vllm) cover all three paths.

Follow-ups outside this PR

Comparing configs/runners.yaml pool prefixes with runners/launch_*.sh: launch_h100-cr.sh and launch_mi325x-tw.sh have no pool, and pool mi300x-tw has no launcher. Those should be reconciled under the same rule but are separate decisions.

No benchmark performance impact, so no perf-changelog.yaml entry.

AI model disclosure

• Model/version: claude-fable-5-1 (Claude Fable 5.1), run through Claude Code.
• Role: investigated the runner/launcher routing, drafted the launcher consolidation, the AGENTS.md section, the equivalence check, and this description. Reviewed and submitted by the PR author.

Related Issue

N/A

Type of Change

• Bug fix
• New feature
• Configuration change
• Documentation update
• Other: launcher refactor, no behavior change intended

Checklist

• I have completed the AI model disclosure and kept it current
• I have tested my changes locally (syntax, shellcheck, path-selection equivalence; not executed on the cluster)
• I have updated documentation if necessary
• perf-changelog entry: not applicable, no performance-affecting change
• /use <run_id> before merging via reuse

🤖 Generated with Claude Code

[github-actions]

Thanks for the contribution!

• Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
• PR verification: Sweeps only run on labeled PRs. Add full-sweep-fail-fast (strongly recommended); use full-sweep-enabled only when matrix jobs should continue after a failure.
• After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

• **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
• **PR 验证：**扫描仅在带有标签的 PR 上运行。强烈建议添加 full-sweep-fail-fast；仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。
• **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

[claude]

I reviewed this PR and didn't find any bugs. Because this consolidates a production CI launcher into a single ~900-line file with three execution paths and has not been run on the cluster yet, a human look would still be worthwhile before merging.

• Diffed the new native-srt, multinode-srt, and single-node function bodies against their old sources in launch_b200-nscale-slurm.sh/launch_b200-nscale-compat.sh — matches the described squash-helper consolidation and check_env_vars relocation.
• Checked the new shared enroot_uri_for_image/import_squash helper against the prior per-path logic; the FQDN-registry splitting mirrors the existing pattern in launch_gb200-nv.sh, but for the multinode-srt path it does change the enroot pull URI for jobs whose main IMAGE is an FQDN registry ref (e.g. the nvcr.io/... image used by dsr1-fp4-b200-dynamo-trt), consistent with the one candidate the automated hunt already flagged.
• Confirmed AGENTS.md's new launcher-routing section and the dropped b200-nscale-compat alias in runtime_settings.sh match the pool-to-launcher mapping in configs/runners.yaml.

Extended reasoning...

Overview

The PR merges runners/launch_b200-nscale-compat.sh into runners/launch_b200-nscale-slurm.sh, restoring a 1:1 pool-to-launcher mapping, and adds a routing rule to AGENTS.md. The new file selects one of three execution paths (native-srt, multinode-srt, single-node) up front and carries each old path's body over as a function, with a new shared enroot_uri_for_image/import_squash pair replacing the two files' separate squash-import logic. runners/runtime_settings.sh drops the now-dead b200-nscale-compat case alias, and a benchmark script comment is reworded to point at the surviving launcher.

Security risks

None of substance — this is CI/build-orchestration shell script for a self-hosted Slurm runner pool, not a service handling external input, auth, or secrets. The one behavior-sensitive area is which enroot/Docker registry a container image is pulled from; a wrong registry could pull an unexpected image, but that is an operational/availability risk (a broken benchmark job) rather than a security exposure, since IMAGE values come from the repo's own trusted config files, not untrusted input.

Level of scrutiny

This warrants more than a rubber stamp: it is a large (779 insertions / 770 deletions), structurally involved refactor of a production CI launcher covering three distinct execution lanes, and the author states it has not been exercised on the cluster (validation is deferred to the next live sweep). Comparing the old and new bodies line-by-line, the carried-over logic matches the PR's own description closely, and the consolidated enroot_uri_for_image helper mirrors an existing, already-used pattern from launch_gb200-nv.sh rather than being novel. However, I found one real, non-trivial behavior difference in the multinode-srt path: previously (in the compat launcher) the main IMAGE was imported via a literal docker://$IMAGE, with no FQDN-registry splitting — only the separately-injected DCGM exporter image got the nvcr.io# substitution. The new consolidated import_squash now applies enroot_uri_for_image to every image, including the main one, so an FQDN main image (e.g. the nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime image used by the live dsr1-fp4-b200-dynamo-trt config in configs/nvidia-master.yaml, which runs on cluster:b200-nscale with multinode: true and framework: dynamo-trt, landing in the multinode-srt path) now resolves to a different enroot URI than before. This is exactly the one candidate already surfaced and investigated by the automated bug hunt; I was not able to independently confirm from the repo alone whether the old or the new URI is the one that actually works against this registry, since that depends on enroot runtime behavior that isn't exercised in this sandbox. Given the change is unverified on the cluster and this ambiguity touches a real, in-use config, a maintainer with cluster access should confirm the import still succeeds for that job before merging.

Other factors

The author's own equivalence testing (840 combinations of path-selection inputs) and the AI disclosure are documented in the PR body, and the diff is broadly consistent with the described mechanical carry-over (removed dead minimaxm2.5 duplicate block, moved check_env_vars calls, one comment reword). There are no CODEOWNERS constraints evident and no outstanding third-party review objections in the timeline provided. The combination of "large, non-trivial infra refactor," "not run on the cluster," and "one plausible but unconfirmed registry-URI behavior change touching a live config" is enough that I'm deferring rather than approving, while still not raising this as a new inline finding since it echoes what the automated hunt already looked at.