-
Notifications
You must be signed in to change notification settings - Fork 3k
test(review): single-encode the diff-read fixture and pin the failed-read gate #9539
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[Suggestion] This new denied-read fixture pins the
!isErrorPartsuccess gate only fordiffToolCalls/diffReads— but the same gate also guardssuccessfulCallArgsandsuccessfulReadFileArgs(transcripts.ts:400-401), and no fixture anywhere asserts that a failed call stays OUT of those two lists. A mutant hoistingsuccessfulCallArgs.push(pending.args)/successfulReadFileArgs.push(pending.args)out of the!isErrorPartbranch ships the entire suite green: "counts only successful tool calls" asserts onlysuccessfulToolCalls === 0(its denied call's args are never inspected), this fixture asserts onlylaunchPrompt/diffToolCalls/diffReads, and the only args-list assertions (~lines 232-235) use all-successful responses. Under that mutant a denied or hallucinatedread_fileof the brief or the findings list leaks into the evidence lists, and the certification atoms that read them —openedBrief(certification.ts:95),readBrief(:110),readFindingsPointer(:128) — certify an agent as having opened/read files it was refused, the exact "naming is not opening" fabrication this module exists to prevent. Verified by probe: with the mutant applied, the entiresrc/commands/reviewdirectory ships green (90 test files, 3816 tests); adding the two assertions below turns this test red under the mutant (expected [ …(4) ] to have a length of 3 but got 4— the denied read's args leaked intosuccessfulCallArgs) and stays green on the correct code. Since the denied read is already present in this fixture, the gate can be pinned right here — add, alongside the existing assertions:中文说明
新增的被拒读取 fixture 只为
diffToolCalls/diffReads钉住了!isErrorPart成功门控 —— 但同一个门控同时守着successfulCallArgs和successfulReadFileArgs(transcripts.ts:400-401),而全部 fixture 中没有任何一处断言失败的调用不会进入这两个列表。变异实验:把successfulCallArgs.push(pending.args)/successfulReadFileArgs.push(pending.args)提出!isErrorPart分支后,整个测试套件全绿通过 —— "counts only successful tool calls" 只断言了successfulToolCalls === 0(其被拒调用的参数从未被检查),本 fixture 只断言launchPrompt/diffToolCalls/diffReads,仅有的 args 列表断言(约 232-235 行)用的全是成功响应。在该变异下,一次被拒或被幻觉出的read_file(读 brief 或 findings 列表)会泄漏进证据列表,读取这些列表的认证原子 ——openedBrief(certification.ts:95)、readBrief(:110)、readFindingsPointer(:128)—— 会把一个被拒绝读取文件的 agent 认证为已打开/已读取,正是本模块要防止的「命名不等于打开」式造假。探针验证:施加变异后整个src/commands/review目录全绿(90 个测试文件、3816 个用例);补上下面两条断言后,该测试在变异下翻红(expected [ …(4) ] to have a length of 3 but got 4—— 被拒读取的参数泄漏进了successfulCallArgs),在正确代码下保持绿色。被拒读取已经存在于本 fixture 中,门控可以直接在这里钉住 —— 在现有断言旁补上:— qwen3.8-max via Qwen Code /review (v0.21.14)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Deferred under the convergence posture — recorded, not declined. This is a real and adjacent gap (the same
!isErrorPartgate guardssuccessfulCallArgs/successfulReadFileArgstoo), but it is one more step of the same coverage tail: #9484 pinned the brief atoms, #9539 pinned the diff-read count and range, and this would pin the two arg lists next. Each is a Suggestion, not a blocker, and the PR chain here has already run well past the ~5-round convergence bar. Landing it as its own tiny PR every time trades review cost against a mutant nobody has shipped; better batched into the next substantive change to this reader than chased on its own. Happy to fold it in whentranscripts.tsis next touched for real.