feat(eval): compare three edit contracts on the DeepSeek Harness - #3067

Closed

Astro-Han wants to merge 33 commits into

feat/eval-deepseek-harness-subjectfrom

feat/eval-edit-contract-arms

Commits on Aug 15, 2026

feat(eval): add a V4A apply_patch tool for the DeepSeek Harness
Astro-Han
committed
6fc993e
feat(eval): compare three edit contracts on the DeepSeek Harness
Astro-Han
committed
587055e
chore(eval): re-pin the DeepSeek Harness toolchain fingerprint
Astro-Han
committed
d5b780f
feat(eval): implement Delete File and Move to in apply_patch
Astro-Han
committed
fa79d5a
chore(eval): re-pin the DeepSeek Harness toolchain fingerprint
Astro-Han
committed
743694e
fix(eval): make apply_patch faithful to Codex's contract
Astro-Han
committed
83538c8
fix(eval): remove the confounds between the three edit-contract arms
Astro-Han
committed
cf0c1c4
fix(eval): apply V4A patches the way Codex does
Astro-Han
committed
b3e5a82
docs(eval): state what the matched read budget does and does not hold
Astro-Han
committed
3651eb8
test(eval): fuzz the apply_patch port against the real codex binary
Astro-Han
committed
e1b1fde
fix(eval): apply V4A patches in Codex's two passes, not one
Astro-Han
committed
bfe0e71
test(eval): run Codex's own apply-patch scenario suite against this port
Astro-Han
committed
455e66f
test(eval): assert the apply_patch surface the model actually sees
Astro-Han
committed
65dfbfe
fix(eval): parse the envelope boundary the way Codex parses it
Astro-Han
committed
25fa09e
docs(eval): describe the read budget by what it actually does
Astro-Han
committed
0853335
docs(eval): state the deviation the patch arm cannot remove
Astro-Han
committed
33e55df
refactor(eval): drop the parent-directory creation no provider needed
Astro-Han
committed
b9b5eea
fix(eval): report an applier defect as a defect, and pin the match passes
Astro-Han
committed
81690a6
fix(eval): keep an insertion a later chunk replaces from surviving
Astro-Han
committed
e1116d0
fix(eval): correct the apply_patch port at the provider seam
Astro-Han
committed
69f3dea
chore(eval): drop the duplicated copy of the upstream scenario suite
Astro-Han
committed
3ef9955
fix(eval): close three uncontrolled differences in the edit-contract arms
Astro-Han
committed
cfb5dd1
docs(eval): correct fourteen false claims about the apply_patch arm
Astro-Han
committed
f438925
fix(eval): read patch targets the way the reference reads them
Astro-Han
committed
df1987f
fix(eval): revert the per-line read cap, which starved the arm it was meant to help
Astro-Han
committed
d3bebac
docs(eval): correct the provenance claim and six other assertions
Astro-Han
committed
50cdcbf
fix(eval): export the plugin path for every arm, not just the one that reads it
Astro-Han
committed
ff2a270
chore(eval): size edit-contract concurrency to the host that runs it
Astro-Han
committed
da13b2c
fix(eval): boot the apply-patch arm
Astro-Han
committed
c5a00fd
fix(eval): keep the tail of an external subject's stderr
Astro-Han
committed
f7c879e
fix(eval): keep the stderr tail out of the result frame
Astro-Han
committed
79be4a1
fix(eval): report the setup error the frame cannot carry
Astro-Han
committed
90d7a82
fix(eval): declare the stderr tail cap before the module runs
Astro-Han
committed
2a8bb2b

Navigation Menu

Uh oh!

feat(eval): compare three edit contracts on the DeepSeek Harness - #3067

feat(eval): compare three edit contracts on the DeepSeek Harness#3067
Astro-Han wants to merge 33 commits into
feat/eval-deepseek-harness-subjectfrom
feat/eval-edit-contract-arms

Commits on Aug 15, 2026

feat(eval): add a V4A apply_patch tool for the DeepSeek Harness

feat(eval): compare three edit contracts on the DeepSeek Harness

chore(eval): re-pin the DeepSeek Harness toolchain fingerprint

feat(eval): implement Delete File and Move to in apply_patch

chore(eval): re-pin the DeepSeek Harness toolchain fingerprint

fix(eval): make apply_patch faithful to Codex's contract

fix(eval): remove the confounds between the three edit-contract arms

fix(eval): apply V4A patches the way Codex does

docs(eval): state what the matched read budget does and does not hold

test(eval): fuzz the apply_patch port against the real codex binary

fix(eval): apply V4A patches in Codex's two passes, not one

test(eval): run Codex's own apply-patch scenario suite against this port

test(eval): assert the apply_patch surface the model actually sees

fix(eval): parse the envelope boundary the way Codex parses it

docs(eval): describe the read budget by what it actually does

docs(eval): state the deviation the patch arm cannot remove

refactor(eval): drop the parent-directory creation no provider needed

fix(eval): report an applier defect as a defect, and pin the match passes

fix(eval): keep an insertion a later chunk replaces from surviving

fix(eval): correct the apply_patch port at the provider seam

chore(eval): drop the duplicated copy of the upstream scenario suite

fix(eval): close three uncontrolled differences in the edit-contract arms

docs(eval): correct fourteen false claims about the apply_patch arm

fix(eval): read patch targets the way the reference reads them

fix(eval): revert the per-line read cap, which starved the arm it was meant to help

docs(eval): correct the provenance claim and six other assertions

fix(eval): export the plugin path for every arm, not just the one that reads it

chore(eval): size edit-contract concurrency to the host that runs it

fix(eval): boot the apply-patch arm

fix(eval): keep the tail of an external subject's stderr

fix(eval): keep the stderr tail out of the result frame

fix(eval): report the setup error the frame cannot carry

fix(eval): declare the stderr tail cap before the module runs