[AgentX]: Use burst phase starts by default / [AgentX]:默认使用阶段起始突发模式 - #2375
Conversation
Allow saturation recipes to request AIPerf burst phase starts without changing the default faithful replay behavior. Validate the environment value and cover enabled, disabled, and invalid command generation. 中文:允许饱和度测试配置按需启用 AIPerf 阶段起始突发,同时保持默认的忠实回放行为不变。新增环境变量校验,并覆盖启用、禁用和非法值的命令生成测试。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 如需进行 PR 验证,请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
Start selected AgentX trajectories together by default so configured concurrency is active within the benchmark window. Preserve faithful recorded phase-start spreading as an explicit AIPERF_BURST_PHASE_STARTS=0 override. 中文:默认同步启动选中的 AgentX 轨迹,使配置的并发度能在基准测试窗口内生效;同时保留 AIPERF_BURST_PHASE_STARTS=0,用于显式恢复按记录时间分散启动。
Update the AIPerf submodule to make burst phase starts the default and remove duplicate InferenceX wrapper plumbing and tests. 中文:更新 AIPerf 子模块,将阶段起始突发设为默认行为,并移除 InferenceX 中重复的封装逻辑和测试。
Point the AIPerf submodule at the cquil11/aiperf-agentx-v1.0 merge commit containing the phase-start default change. 中文:将 AIPerf 子模块更新到 cquil11/aiperf-agentx-v1.0 分支中包含阶段起始默认行为变更的合并提交。
|
Claude finished @cquil11's task in 2m 26s —— View job PR Review
LGTM - no blocking issues found. The diff is exactly the one-line
Note: I couldn't inspect the AIPerf commit contents directly (submodule not initialized in this environment and network access to the aiperf repo is restricted), so the behavioral review of the default change itself rests on the merged upstream review in SemiAnalysisAI/agentx-harness#22 and the reported validation (758 AIPerf tests, 224 matrix tests). |
There was a problem hiding this comment.
LGTM - this is a one-line submodule pointer bump (utils/aiperf to 208125ac) pulling in an already-merged, reviewed upstream change that flips burst_phase_starts to default True. The one finding is a doc-only nit (stale docstring text) inside the submodules own commit history with no functional impact.
Extended reasoning...
Overview
The entire InferenceX-side diff is a single line: the utils/aiperf submodule pointer moves from f5ae8134 to 208125ac, the merge commit for cquil11/aiperf-agentx-v1.0 (upstream PR SemiAnalysisAI/agentx-harness#22). There is no InferenceX wrapper code, config, or test file touched - the behavioral change (Agentic Replay warmup/profiling phase starts default to burst instead of spread) lives entirely inside the aiperf submodule.
Security risks
None. This is a benchmarking-tool timing/dispatch behavior change with no auth, crypto, injection, or data-exposure surface.
Level of scrutiny
Low-to-moderate is appropriate: the InferenceX-visible change is a mechanical submodule bump, so I read into the submodule itself to verify the underlying commit (fb39d0df) actually does what the PR description claims. I confirmed the config default flip in loadgen_config.py (False to True), the corresponding docstring update in docs/cli-options.md, and that the dispatch logic in agentic_replay.py (spread = not self._burst_phase_starts, and the burst/spread branches in _execute_warmup and _dispatch_snapshot_for_profiling) correctly implements the new default - no functional bug there. The behavior is escapable via burst_phase_starts: false for anyone who needs the old faithful-replay timing.
Other factors
The one flagged issue (stale spread-is-default wording left in two docstrings/comments in the same file, contradicting the now-accurate module-level docstring) is cosmetic, pre-existing in the upstream commit being pulled in, and does not affect runtime behavior - the actual branch conditions were updated correctly everywhere checked. It does not warrant blocking this bump; a follow-up doc fix in the upstream aiperf repo is sufficient. The PR description also documents concrete motivation (concurrency never reaching target under spread mode, evidenced by large recorded offsets at concurrency 32/256) and reports both AIPerf's and InferenceX's test suites passing.
| @@ -1 +1 @@ | |||
| Subproject commit f5ae8134f863993e412d350e406fbcacfe714a7a | |||
| Subproject commit 208125aca87a438e43e56517e8a3e5096f8c9281 | |||
There was a problem hiding this comment.
🟡 The aiperf submodule bump (fb39d0df) flips burst_phase_starts to default True and updates the module-level docstring in agentic_replay.py accordingly, but two other docstrings in the same file — _execute_warmup's "Dispatch timing" docstring (~line 496) and the inline comment in _dispatch_snapshot_for_profiling (~line 1499) — still describe spread as the default and burst as the opt-in flag, now contradicting actual behavior. This is a pre-existing issue in the upstream aiperf commit being pulled in, doc-only with no functional impact, and would need a follow-up fix + re-bump in the aiperf repo.
Extended reasoning...
What the bug is: Commit fb39d0df ("refactor(agentic): burst phase starts by default") in the bumped utils/aiperf submodule flips burst_phase_starts from False to True (see loadgen_config.py:278) and correctly updates the module-level docstring and the __init__ comment in src/aiperf/timing/strategies/agentic_replay.py to describe burst as the new default. However, two other docstrings in the same file that describe the same flag were left untouched and now say the opposite of what's true.
Where it manifests:
_execute_warmup's "Dispatch timing" docstring (~line 496-507) still reads:Default (spread): aligned GLOBALLY...and frames--burst-phase-startsas the override that changes behavior away from the default.- The inline comment inside
_dispatch_snapshot_for_profiling(~line 1499-1501) still reads:Spread (default): t0 = 0 ... Burst (--burst-phase-starts): t0 = the lane's min offset.
Both explicitly label "spread" as the default, which was true before this commit but is now false — the default is burst.
Why the existing update missed these: The diff for fb39d0df only touched the module docstring (lines ~20-40) and the __init__ comment (lines ~215-221), which are the two most visible places describing the flag. It missed that the same "default vs. override" framing was duplicated verbatim in two method-level docstrings further down in the file, so those got left behind when the semantics flipped.
Impact: This is purely a documentation/comment inconsistency — the actual dispatch logic in both methods is correct and unaffected (the code paths still correctly branch on self._burst_phase_starts). No runtime behavior changes. The only impact is that a future reader of _execute_warmup or _dispatch_snapshot_for_profiling in isolation would be misled about which behavior is the current default, since these method-level docs now contradict the accurate module-level docstring in the same file.
Proof (step-by-step):
- Before
fb39d0df:burst_phase_startsdefaulted toFalse(spread). Module docstring,__init__comment,_execute_warmupdocstring, and_dispatch_snapshot_for_profilingcomment all consistently said "spread is default." fb39d0dfchangesloadgen_config.py's default toTrue(burst) and updates the module docstring (~line 24: "By default the priming requests BURST together...") and__init__comment (~lines 215-216: "Real loadgen configs default to burst...").- The commit's diff (visible in the verifier logs) shows no changes to the
Default (spread):string in_execute_warmupor theSpread (default):string in_dispatch_snapshot_for_profiling. - Reading those two methods today:
_execute_warmupsays spread is default while the module docstring 60 lines above it says burst is default — a direct, verifiable contradiction within the same file.
How to fix: In the upstream aiperf repo, swap the "default"/"override" labeling in both the _execute_warmup docstring and the _dispatch_snapshot_for_profiling inline comment to match the new burst-by-default semantics (mirroring the wording already used in the module docstring), then re-bump the utils/aiperf submodule pointer in InferenceX. This doesn't need to block the current PR since it's doc-only and pre-existing in the commit being pulled in — a quick follow-up is sufficient.
* feat(agentx): add Kimi K3 GB200 day-0 support Add srt-slurm TP, TEP, and DEP AgentX profiles plus GB200 runner routing and recipe overlay plumbing. 中文:新增 Kimi K3 GB200 首日 AgentX 支持,包括 srt-slurm TP、TEP、DEP 配置,以及 GB200 运行器路由和配方覆盖逻辑。 * chore: link GB200 Kimi K3 changelog Record the draft pull request URL in the append-only benchmark changelog entry. 中文:在仅追加的基准测试变更日志条目中记录草稿 PR 链接。 * fix(agentx): align Kimi K3 DEP metadata Represent the throughput arm as TP1 x DP16 = EP16 in the master configuration, matching the vLLM recipe. 中文:将吞吐量配置修正为 TP1 × DP16 = EP16,与 vLLM 配方保持一致。 * perf(gb200): load Kimi K3 from local NVMe Use the pre-staged /mnt/numa1/models/Kimi-K3 checkpoint on every GB200 compute node while retaining shared-filesystem orchestration and caches. 中文:在每个 GB200 计算节点上使用预置的 /mnt/numa1/models/Kimi-K3 检查点,同时保留共享文件系统上的编排与缓存。 * chore: simplify Kimi K3 performance config Keep the Day-0 changelog focused on the vLLM recipe performance points and use the model default context length. 中文:精简 Kimi K3 首日性能配置,仅保留与 vLLM 配方一致的性能测试点,并使用模型默认上下文长度。 * perf(gb200): size Kimi K3 AgentX sweep from HBM Move latency and balanced profiles to 16-GPU TP/TEP, use TP2 x DP8 for EP16 throughput, and extend the throughput concurrency sweep through 256. 中文:根据 GB200 HBM 容量调整 Kimi K3 AgentX 扫描。将延迟与均衡配置扩展为 16 GPU TP/TEP,吞吐配置改为 TP2 × DP8(EP16),并将吞吐并发扫描扩展至 256。 * fix(gb200): drop unsupported Kimi K3 worker parsers The Dynamo vLLM worker entrypoint does not accept vllm serve tool-call or reasoning parser flags; agentic request parsing belongs to the Dynamo frontend. 中文:移除 Dynamo vLLM worker 不支持的 Kimi K3 工具调用与推理解析参数;AgentX 请求解析由 Dynamo frontend 负责。 * fix(gb200): let srt-slurm select collective interfaces Remove literal shell placeholders from the vLLM worker environment so Gloo and NCCL use the interfaces selected by the srt-slurm launcher. 中文:移除 vLLM worker 环境中的字面量 shell 占位符,由 srt-slurm launcher 为 Gloo 与 NCCL 选择网络接口。 * fix(gb200): launch Kimi K3 TP2 DP8 per node Patch the pinned agentic srt-slurm checkout to launch one vLLM process per GB200 node, with two local DP ranks and TP2 per rank. Export every per-node metrics endpoint to AIPerf. 中文:修补 AgentX 固定版本的 srt-slurm,使每个 GB200 节点启动一个 vLLM 进程,每个进程承载两个本地 DP rank,并为每个 rank 使用 TP2;同时将所有节点的 metrics endpoint 传给 AIPerf。 * fix(gb200): rebalance Kimi K3 EP16 for HBM Use TP4 x DP4 for the throughput-oriented EP16 profile after TP2 x DP8 exceeded GB200 HBM during MXFP4 weight loading. 中文:TP2 × DP8 在 MXFP4 权重加载阶段超过 GB200 HBM 容量,因此将吞吐优先的 EP16 配置调整为 TP4 × DP4。 * fix(gb200): use native srt-slurm TPDP support Pin a tested srt-slurm revision that natively groups TP GPUs within each DP rank and exports every DP metrics endpoint. Remove the vendored patch and reserve enough HBM for one native 1M-context request. 中文:固定经过测试的 srt-slurm 版本,原生支持每个 DP rank 内的 TP GPU 分组,并导出全部 DP metrics endpoint。删除仓库内补丁,同时为原生 1M 上下文请求预留足够 HBM。 * docs(perf): correct Kimi K3 throughput topology Record the tested TP4 x DP4 = EP16 throughput profile. 中文:记录经过测试的 TP4 x DP4 = EP16 吞吐配置。 * fix(gb200): use official Kimi K3 Dynamo runtime Use NVIDIA’s Kimi K3 runtime with native tokenizer, parser, FP8 KV-cache, and FlashInfer support. Keep the model-default 1M context and remove the vLLM-mutating setup script.\n\n中文:改用 NVIDIA 官方 Kimi K3 Dynamo 运行时,启用原生 tokenizer、解析器、FP8 KV 缓存和 FlashInfer 支持。保留模型默认的 1M 上下文,并移除会修改 vLLM 文件的环境配置脚本。 * fix(gb200): align Kimi K3 runtime metadata Match identity verification and result metadata to the Dynamo 1.3.0 package reported by the official Kimi K3 runtime image.\n\n中文:将身份校验与结果元数据对齐到官方 Kimi K3 运行时镜像实际报告的 Dynamo 1.3.0 包版本。 * fix(gb200): reserve Kimi K3 prefill workspace Use the official Kimi K3 Dynamo 0.92 GPU memory utilization so native 1M-context MLA prefills retain transient workspace without changing max model length.\n\n中文:采用官方 Kimi K3 Dynamo 配方的 0.92 GPU 显存利用率,为原生 1M 上下文 MLA 预填充保留临时工作区,且不修改最大模型长度。 * fix(config): report Kimi K3 TP16 as EP16 Apply vLLM EP_SIZE = TP_SIZE × DP_SIZE in the master metadata for the DP1 latency profile.\n\n中文:在 DP1 延迟配置的主配置元数据中应用 vLLM 的 EP_SIZE = TP_SIZE × DP_SIZE 规则。 * fix(gb200): stage srt AgentX aggregates Copy per-concurrency AgentX aggregate JSON files from the compute-visible shared workspace back into the GitHub workspace, and cover success and missing-result behavior with regression tests. 中文:将各并发度的 AgentX 聚合 JSON 从计算节点可见的共享工作区复制回 GitHub 工作区,并通过回归测试覆盖成功复制和结果缺失场景。 * fix(agentx): burst Kimi K3 saturation starts Enable AIPerf burst phase starts only for the Kimi K3 GB200 saturation recipes so warmup and profiling reach the requested concurrency within the benchmark window. Inter-turn replay timing remains unchanged. 中文:仅在 Kimi K3 GB200 饱和度测试配置中启用 AIPerf 阶段起始突发,使预热和正式测试能在基准测试窗口内达到目标并发度,同时保持轮次间回放时序不变。 * refactor(kimik3): depend on shared phase starts Remove the duplicated AIPerf command-builder hook and Kimi-specific opt-ins. The shared default is tracked in InferenceX PR #2375. 中文:移除重复的 AIPerf 命令构建逻辑和 Kimi 专用开关,统一依赖 InferenceX PR #2375 提供的共享默认行为。 * fix(agentx): extend Kimi K3 DEP warmup drain Allow the full-context DEP saturation sweep up to 3600 seconds to drain accelerated warmup requests. The grace period exits early when the servers become idle. 中文:将 Kimi K3 DEP 全上下文饱和度测试的加速预热排空上限延长至 3600 秒;服务空闲后会提前结束等待。 * chore(runners): register fourth GB200 worker Add gb200-nv_3 to the GB200 cluster runner pool so the Kimi K3 sweep can use the capacity provisioned for this bring-up. 中文:将 gb200-nv_3 注册到 GB200 集群运行器池,使 Kimi K3 扫描能够使用本次适配新增的容量。 * feat(agentx): add GB200 Kimi K3 CPU KV offload Add a high-concurrency DEP16 SimpleCPUOffloadConnector arm at c128/c192/c256/c384 and size the pinned CPU KV tier to 128 GiB per TP rank. Correct the pure TP16 result identity to EP1 so its c8 result does not collide with TEP16. 中文:为 GB200 Kimi K3 AgentX 新增 DEP16 SimpleCPUOffloadConnector 高并发测试,覆盖 c128/c192/c256/c384,并为每个 TP rank 配置 128 GiB 主机 DRAM KV 缓存。同时将纯 TP16 的结果标识修正为 EP1,避免其 c8 结果与 TEP16 冲突。 * feat(kimik3): use upstream vLLM with DSpark on GB200 Switch all GB200 AgentX recipes to vllm/vllm-openai:kimi-k3, install Dynamo 1.3.0 in-container, and enable DSpark level 2 at the prescribed golden acceptance length. Consolidate the stacked changelog records into one PR #2404 entry.\n\n中文:将所有 GB200 AgentX 配置切换到 vllm/vllm-openai:kimi-k3,在容器内安装 Dynamo 1.3.0,并按规定的黄金接受长度启用 DSpark level 2;同时将堆叠分支中的性能变更日志合并为一条 PR #2404 记录。 * fix(kimik3): pin K3-compatible Dynamo preview Install the official v1.4.0-kimi-k3-dev.1 preview from its immutable release commit so the upstream vLLM image has Kimi K3 reasoning and tool-call support. 中文:固定使用支持 Kimi K3 的 Dynamo 预览版。通过不可变发布提交安装官方 v1.4.0-kimi-k3-dev.1,使上游 vLLM 镜像具备 Kimi K3 推理与工具调用支持。 * fix(kimik3): bridge DSpark mask metadata for Dynamo Create an ephemeral view of the official Kimi K3 DSpark checkpoint that maps its mask_token_id to the pard_token field expected after Dynamo serialization. Keep the upstream vLLM image and checkpoint weights unchanged.\n\n中文:为官方 Kimi K3 DSpark 检查点创建临时兼容视图,将 mask_token_id 映射到 Dynamo 序列化后所需的 pard_token 字段;保持上游 vLLM 镜像与检查点权重不变。 * fix(kimik3): use model runner v2 for DSpark Enable MRV2 in all GB200 Kimi K3 AgentX recipes so DSpark uses its supported loader instead of the legacy embedding-sharing path.\n\n中文:在所有 GB200 Kimi K3 AgentX 配置中启用 MRV2,使 DSpark 使用受支持的加载路径,避免旧版 embedding sharing 路径。 * fix(kimik3): preserve full context on DEP Raise the GB200 DEP and SimpleCPU-offload recipes to the official 0.95 GPU memory utilization so MRV2 can allocate the 16.9 GiB required for one full 1M-token request. 中文:将 GB200 DEP 与 SimpleCPU 卸载配置的 GPU 显存利用率提高到官方建议的 0.95,使 MRV2 能为单个完整 100 万 token 请求分配所需的 16.9 GiB KV 缓存。 * fix(kimik3): reserve DEP graph memory Tune GB200 DEP GPU memory utilization to 0.94: the measured 0.95 setting cleared the full-context KV admission gate but exhausted CUDA memory during MRV2 graph capture. 中文:将 GB200 DEP 的 GPU 显存利用率调至 0.94;实测 0.95 虽能通过完整上下文的 KV 缓存准入检查,但会在 MRV2 CUDA Graph 捕获阶段耗尽显存。 * fix(kimik3): use piecewise DEP graphs Keep MRV2 PIECEWISE CUDA graphs for GB200 DEP while disabling the FULL capture phase that OOMs after full-context KV allocation. This follows existing GB200 AgentX recipe practice and avoids eager execution. 中文:GB200 DEP 保留 MRV2 PIECEWISE CUDA Graph,同时禁用在完整上下文 KV 缓存分配后触发显存溢出的 FULL 捕获阶段。该设置沿用仓库现有 GB200 AgentX 配置,并避免退回纯 eager 执行。 * fix(kimik3): stabilize DSpark DEP MoE Use the supported FlashInfer TRT-LLM MoE backend for Kimi K3 DSpark DEP after DeepGEMM grid barriers timed out under MRV2 with one-sided all-to-all. Preserve the default 1M context, DEP16 topology, DSpark, and the one-sided FlashInfer all-to-all path. 中文:Kimi K3 DSpark DEP 在 MRV2 与单边 FlashInfer all-to-all 组合下触发 DeepGEMM 网格屏障超时,因此改用受支持的 FlashInfer TRT-LLM MoE 后端;保持默认 1M 上下文、DEP16 拓扑、DSpark 和单边 all-to-all 不变。 * fix(agentx): reduce Kimi K3 DEP load memory Use the standard safetensors loader for GB200 DEP recipes so FlashInfer MoE and DSpark do not exhaust transient HBM in the fastsafetensors GPU-staging path. 中文:GB200 DEP 配置改用标准 safetensors 加载器,避免 FlashInfer MoE 与 DSpark 在 fastsafetensors GPU 暂存阶段耗尽瞬时 HBM。 * fix(agentx): right-size Kimi K3 DEP graphs Limit regular DEP captures to 64 sequences per engine and offload captures to 96, matching the highest configured global concurrencies under DP4 while retaining the default 1M context. 中文:按 DP4 下配置的最高全局并发量,将常规 DEP 的每引擎 CUDA Graph 捕获上限设为 64,将卸载配置设为 96;保留默认 1M 上下文,并为 FlashInfer MoE 运行时工作区释放显存。 * fix(agentx): report pinned Dynamo version Align all Kimi K3 GB200 recipe identities and the master-config router label with the 1.3.0 version reported by the pinned K3-enabled source checkout. 中文:将所有 Kimi K3 GB200 配方身份信息及主配置中的路由版本统一为固定 K3 源码构建实际报告的 1.3.0。 * fix(agentx): reduce offload graph residency Capture even DSpark sequence counts through the c384 DP4 steady state. This preserves every configured steady-state batch while releasing the HBM needed by the FlashInfer MXFP4 MoE workspace. 中文:按偶数序列数捕获 DSpark CUDA Graph,覆盖 c384/DP4 稳态批次;保留所有已配置稳态点,同时释放 FlashInfer MXFP4 MoE 工作区所需显存。 * fix(kimik3): extend offload sweep allocation Allow the full-context high-concurrency AgentX warmup and one-hour profile to complete by using the GB200 batch partition maximum. 中文:将 Kimi K3 GB200 Simple CPU KV 卸载配方的 Slurm 时限延长至 12 小时,使全上下文高并发 AgentX 预热和一小时性能采集能够完整结束。 * perf(kimik3): report prompt cache reads Enable vLLM prompt-token details on every GB200 Kimi K3 AgentX profile so AIPerf can record per-request cached prompt tokens and aggregate cache-read rates. 中文:为所有 GB200 Kimi K3 AgentX 性能配置启用 vLLM prompt-token details,使 AIPerf 能记录每个请求的缓存提示词 token 数及汇总缓存读取率。 * fix(agentx): align GB200 replay timing Pin the same AIPerf warmup-handoff revision used by the Qwen B200 and B300 AgentX PRs. Add a recipe-scoped 300-second per-trace idle-gap cap for all Kimi K3 GB200 profiles while retaining full model context and cached-token reporting. 中文:将 AIPerf 固定到 Qwen B200/B300 AgentX PR 使用的同一预热交接版本。为所有 Kimi K3 GB200 性能配置添加仅配方生效的 300 秒单轨迹空闲间隔上限,同时保留完整模型上下文与缓存 token 上报。 * fix(agentx): let Dynamo report cached prompt tokens Remove the native vLLM OpenAI frontend flag that Dynamo rejects. Dynamo already emits prompt_tokens_details.cached_tokens from RequestOutput for each response. 中文:移除 Dynamo 不支持的原生 vLLM OpenAI 前端参数。Dynamo 已会从每个 RequestOutput 生成 prompt_tokens_details.cached_tokens。 * fix(agentx): align AIPerf with Qwen frontiers Pin the same AIPerf warmup-handoff revision used by the B200 and B300 Qwen AgentX PRs so baseline parents survive into profiling. 中文:将 AIPerf 固定到 B200/B300 Qwen AgentX PR 使用的同一预热交接版本,确保基线父请求能够正确延续到性能分析阶段。 * fix(agentx): separate live and final error gates Allow the Kimi K3 TP recipe to collect a larger live sample before aborting while preserving the existing 10% post-run validity threshold. 中文:为 Kimi K3 TP 配置分离实时中止阈值与最终校验阈值,允许低并发轨迹先积累更充分的实时样本,同时保持现有 10% 运行后有效性门槛。 * fix(agentx): allow full-context sweeps to finish Give multi-node AgentX jobs a 13-hour workflow envelope and the Kimi K3 regular DEP recipe a matching 12-hour Slurm allocation so the canonical ten-request-per-lane warmup and one-hour profile can complete without reducing the 1M context. 中文:将多节点 AgentX 工作流时限扩展到 13 小时,并将 Kimi K3 常规 DEP 的 Slurm 时限同步为 12 小时,使标准的每通道十请求预热和一小时性能采集能够在不缩短 1M 上下文的前提下完成。 * chore(agentx): bump AIPerf harness Pin the GB200 Kimi K3 sweep to the same updated AgentX harness commit used by the B200 and B300 Qwen sweeps. 中文:将 GB200 Kimi K3 扫描固定到 B200 和 B300 Qwen 扫描使用的同一版 AgentX 测试框架提交。 * perf(gb200): reuse Lustre-backed runner workspace Skip the redundant Lustre-to-Lustre AgentX workspace copy when the GitHub runner checkout already resides on Lustre. Preserve staging as a fallback for node-local runner workspaces.\n\n中文:当 GitHub runner 检出目录已位于 Lustre 时,跳过冗余的 AgentX Lustre 到 Lustre 工作区复制;对于节点本地 runner 工作区,保留共享存储暂存回退路径。 * fix(gb200): isolate InferenceX Slurm jobs Namespace GB200 allocations to prevent cross-repository runner cleanup from cancelling active InferenceX jobs. Limit generic Slurm cleanup to the current user, clean up the exact submitted job on exit, and accept results already present in a directly mounted runner workspace.\n\n中文:为 GB200 Slurm 作业添加 InferenceX 命名空间,避免其他仓库的 runner 清理误取消正在运行的 InferenceX 作业;通用 Slurm 清理仅作用于当前用户,退出时按精确作业 ID 清理,并支持结果已位于直接挂载 runner 工作区的场景。 * fix(gb200): preserve namespaced Slurm job name Override RUNNER_NAME only for srtctl submission because srtctl gives the GitHub runner environment precedence over the recipe name. This keeps rendered GB200 allocations isolated from other repositories.\n\n中文:仅在提交 srtctl 作业时覆盖 RUNNER_NAME,因为 srtctl 会优先使用 GitHub runner 环境变量而忽略 recipe 名称;由此确保渲染后的 GB200 Slurm 作业继续使用 InferenceX 命名空间,与其他仓库隔离。 * chore(agentx): bump AIPerf timing harness Advance the AgentX v1 harness to abf55f90 so profiling preserves flattened trace timing across warmup handoff and idle watchdogs remain active across dependency barriers. 中文:将 AgentX v1 基准测试工具升级到 abf55f90,使 profiling 在 warmup 交接后保留展平后的 trace 时序,并确保依赖屏障期间 idle watchdog 持续生效。 * fix(agentx): remove duplicate trace idle cap plumbing Keep the main-branch trace idle cap implementation and retain only the distinct live request-error threshold override needed by the GB200 TP16 recipe. 中文:移除重复的轨迹空闲间隔上限逻辑,沿用 main 分支实现;仅保留 GB200 TP16 配方所需的实时请求错误率阈值覆盖。 * Update perf-changelog.yaml
Summary
utils/aiperfsubmodule fromf5ae8134to208125ac, the merge commit oncquil11/aiperf-agentx-v1.0.burst_phase_starts: falserestores spread phase starts.Merged AIPerf change: SemiAnalysisAI/agentx-harness#22
Discovered while bringing up Kimi K3 GB200 AgentX in #2354. Recorded initial offsets reached 56,048.7 seconds at concurrency 32 and 252,825.1 seconds at concurrency 256, so spread phase starts could prevent the requested concurrency from becoming active during the run.
Validation
中文说明
utils/aiperf子模块从f5ae8134更新到cquil11/aiperf-agentx-v1.0分支上的合并提交208125ac。burst_phase_starts: false可恢复分散式阶段启动。已合并的 AIPerf 变更:SemiAnalysisAI/agentx-harness#22
该问题是在 #2354 的 Kimi K3 GB200 AgentX 启动调试中发现的。并发度为 32 和 256 时,记录的初始偏移分别达到 56,048.7 秒和 252,825.1 秒,分散式阶段启动可能导致目标并发度在测试期间始终无法生效。
验证