Repository navigation
fix(klaud): freeze AgentX datasets and evals so baseline deltas render / fix(klaud)：冻结 AgentX 数据集与评测以显示基线差值 - #4390
Merged
Merged
Conversation
AgentX baselines never recorded their replay dataset, so every AgentX delta rendered N/A. The planner now reads each producer run's dataset slug from benchmark-siblings and builds baseline evals from the public evaluations of verified producer runs, keyed like final artifacts. Points normalize loader names and HF IDs to the dashboard slug; agent reports must name it. Also: infrastructure retries numbered 1-2, report schema errors name the failing field, eval samples no longer show N/A/old, and the prompt stops asking agents to hand-build baseline evals.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AgentX baselines never recorded their replay dataset, so every AgentX Klaud report showed
Δ N/A: dataset/scenario/statistic mismatch./api/v1/benchmark-siblingsand freezes baseline evals from verified producer runs in/api/v1/evaluations, keyed byeval_identitieslike final artifacts.Point.datasetnormalizes loader names and HF IDs to the dashboard slug; hand-written attempts must name it.reportschema errors name the failing field; eval samples without a baseline show only the count.Verified by re-resolving baselines for #4378, #4369, #4323, #4370, #4278 against live APIs: datasets resolve and baseline eval keys match attempt keys. Existing open PRs keep their frozen baselines.