Repository navigation
fix(agentx): default to uncapped traces with explicit 256k exceptions / 默认使用完整轨迹并显式列出 256k 例外 - #4287
Conversation
默认使用完整轨迹数据集，仅为明确列出的模型系列保留 256k 限制，并保持显式覆盖行为。
|
Generally most InferenceX submissions don't need to add changes to the infx package. On the off chance they do, generally the diff is small. Please prompt your agent to clean your PR to keep your diff to only necessary & minimal changes too. We have an increased code quality bar for changes to the infx package. 中文大多数 InferenceX 提交通常不需要修改 infx 包。即使确实需要，改动通常也很小。请让你的 agent 清理 PR，使改动只包含必要且最小的内容。infx 包的改动有更高的代码质量要求。 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Apart from the inline comment, I also checked whether flipping the default changes the corpus for any configured benchmark. Every model-prefix in the repo's YAML configs (dsr1, qwen3.5, qwen3.8next, musespark, dsv4, dsv41flash, glm5.2, glm5.3, kimik3, minimaxm3) still gets the same corpus as before, because dsv41flash matches dsv4. That means skipping the perf-changelog entry looks consistent.
Extended reasoning...
The change flips the corpus allowlist in traces.py from UNCAPPED_FAMILIES to CAPPED_FAMILIES and updates two tests and the standalone doc. I listed every model-prefix value in the checked-in configs and compared what the old and new resolve() return for each one; none of them changes.
| assert result.returncode == 0, result.stderr | ||
| assert _events(tmp_path) == [ | ||
| f"hf download --repo-type dataset {traces.LOADERS[traces.CAPPED]}", | ||
| f"hf download --repo-type dataset {traces.LOADERS[traces.UNCAPPED]}", |
There was a problem hiding this comment.
🟡 nit (optional): Two updated expectations still build the expected corpus from traces constants, so they pass even if those constants are wrong. sweep:traces\.LOADERS\[traces\.|"--public-dataset", traces\. Example: test_agentic_command.py:429 expects traces.LOADERS[traces.UNCAPPED]. test_agentic_replay.py:43 expects traces.UNCAPPED. Fix: hard-code semianalysisai/cc-traces-weka-062126 and semianalysis_cc_traces_weka_062126, as the new parametrized test already does.
Why this was flagged
AGENTS.md:94 and AGENTS.md:103 say expected values must be worked out by hand and never taken from the implementation or its helpers. The diff rewrites test_agentic_command.py:429 to expect hf download --repo-type dataset {traces.LOADERS[traces.UNCAPPED]}. It rewrites test_agentic_replay.py:43 to expect "--public-dataset", traces.UNCAPPED. If someone swaps or mistypes the UNCAPPED or LOADERS values in traces.py, the shim downloads the wrong dataset and replay gets the wrong --public-dataset, but both tests still pass. The new test_corpus_is_uncapped_unless_explicitly_capped_or_pinned in the same diff already uses hand-written literals, so it shows the intended pattern.
Verification: Trigger: someone changes or mistypes the UNCAPPED/LOADERS constants in traces.py. test_agentic_command.py:429 and test_agentic_replay.py:43 still build their expected value from the module under test. AGENTS.md:103 requires "expected values worked out independently by hand, never derived by calling the implementation or its helpers". The pattern predates this PR.
移除 Muse Spark 的 256k 例外，更新测试和文档，并记录七个受影响配置的性能变更。
1 participant
Summary
AgentX currently requires a model family to opt into the full trace corpus. Forgetting to update that allowlist silently selects the 256k-capped workload for a new model.
Default to the full corpus and replace
UNCAPPED_FAMILIESwith explicitCAPPED_FAMILIES:dsr1,qwen3.5, andqwen3.8next.WEKA_LOADER_OVERRIDEstill takes precedence in either direction.Muse Spark Open now uses the full corpus instead of the 256k-capped workload. One performance-changelog entry covers all seven affected Muse Spark config keys. Other existing model families keep their current corpus. The GPU sweep has not been run.
The replay tests cover the unlisted-model default, capped-family matching, loader/dataset agreement, and both override directions. The standalone guide documents the wrapper's selection policy.
A new model that needs the 256k workload must now explicitly select it through the capped-family list or a recipe override.
AI model disclosure
PR checklist and change type
Type of Change
Checklist
python -m infx.matrix.changed --base origin/mainlists affected config keys, I have appended one new entry covering them to the physical end ofinferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entriesinferencex-e2e/perf-changelog.yamlentry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs):full-sweep-fail-fast(recommended),full-sweep-enabled, ornon-canary-full-sweep-enabled. Optional modifiersall-evals,evals-only, andagentx-fastrequire a primary label; the last two block reuse while applied. Awaiting approval to start the GPU sweep.OWNER/MEMBER/COLLABORATOR) has commented/use <run_id>(or the legacy/reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.中文
AgentX 目前要求将模型系列加入白名单后才会使用完整轨迹数据集。新增模型时如果忘记更新白名单，就会静默使用截断至 256k 的工作负载。
本 PR 将完整数据集设为默认值，并用显式的
CAPPED_FAMILIES替换UNCAPPED_FAMILIES，仅为dsr1、qwen3.5和qwen3.8next保留 256k 数据集。WEKA_LOADER_OVERRIDE仍可优先指定任一数据集。Muse Spark Open 从 256k 数据集切换到完整数据集。一条性能变更日志涵盖全部七个受影响的 Muse Spark 配置键，其他现有模型系列的数据集保持不变。GPU sweep 尚未运行，等待批准后再启动。
回放测试覆盖未列出模型的默认行为、受限模型系列的前缀匹配、加载器与数据集的一致性，以及两个方向的显式覆盖。独立运行指南补充了包装命令的数据集选择规则。
后续新增模型如需使用 256k 工作负载，必须显式加入受限模型系列列表，或在配方中指定覆盖值。本次改动包含错误修复和文档更新。
AI 模型：GPT-6 Astra Ultrafast，通过 Perplexity Computer 完成仓库检查、实现、测试、文档和 PR 准备。