[pipeline-dcdad92d][slice-11/12] Full eval suite (four datasets... - #206

Merged

jwbron merged 73 commits into

mainfrom

egg/pipeline-dcdad92d/slice-11

Jul 8, 2026

Commits on Jul 3, 2026

implement(documenter): R1 single-body submission + E2 pr-context staging (slice-1)

authored and
jwbron
committed
441f63a
implement(coder): R8(a) versioned structured finding schema + validator

authored and
jwbron
committed
c2279b6
implement(tester): unit tests for R8(a) finding schema/validator (task-1-4)

authored and
jwbron
committed
6a85827
implement(tester): unit tests for R8(a) finding schema/validator (task-1-4)

authored and
jwbron
committed
107e956
Persist BRC history for slice-1 (#2548)

egg-orchestrator
authored and
jwbron
committed
8826894
slice-1: prettier fixes + changeset
jwbron
committed
c02da14
slice-1: drop BRC history artifacts from slice PR
jwbron
committed
b119a01

Commits on Jul 6, 2026

review: address slice-1 feedback (empty body, lib distribution, comments)
jwbron
committed
b9508a6

Commits on Jul 7, 2026

review: drop lib install step; scripts run via npx tsx
jwbron
committed
123a10f
implement(coder): R8(b) computed verdict + R8(c) Conventional-Comment rendering (slice-2)

authored and
jwbron
committed
e637ece
implement(tester): R8(b)/R2 verdict truth-table + R8(c) rendering snapshot tests (task-2-4)

authored and
jwbron
committed
8819339
implement(tester): prettier-format task-2-4 test files (lint gate)

authored and
jwbron
committed
545e711
implement(coder): prettier 2.6.2 lint fixes for slice-2 + finding-schema (tester NACK v1)

authored and
jwbron
committed
40efb53
Persist BRC history for slice-2 (#2548)

egg-orchestrator
authored and
jwbron
committed
f618d2a
slice-2: changeset
jwbron
committed
6058d86
slice-2: drop BRC history artifacts from slice PR
jwbron
committed
da5b450
review: address slice-2 feedback (verdict precedence, hold UX, empty bodies)
jwbron
committed
0950bf2
implement(documenter): wire deterministic router into Step 3; drop reviewer-mapper (task-3-3)

authored and
jwbron
committed
6cfe3e3
implement(documenter): fix router invocation path to lib/router.ts (reviewer_code NACK v1)

authored and
jwbron
committed
7959787
implement(coder): R10 deterministic router + tier-scaled budget (slice-3)

authored and
jwbron
committed
d96789d
implement(tester): R10 router unit tests — classification, lens, tier, budget, floor (task-3-4)

authored and
jwbron
committed
3f919f3
implement(coder): router CLI entrypoint + routing.json serialization (slice-3 v2)

authored and
jwbron
committed
f3130ec
Persist BRC history for slice-3 (#2548)

egg-orchestrator
authored and
jwbron
committed
a6984f1
implement(tester): cover router v2 serialization + CLI (toRoutingJson/tierFromDisplay/runCli)

authored and
jwbron
committed
c41a812
slice-3: replace satisfies with prettier-2-compatible check, changeset
jwbron
committed
cb8d714
slice-3: drop BRC history artifacts from slice PR
jwbron
committed
29f2b6f
review: address slice-3 feedback (consumer-owned routing config)
jwbron
committed
abdf46c
review: last-match-wins tier precedence; move ROUTING format doc to README
jwbron
committed
06cbb2e
implement(documenter): E1/E3/E5/E6/E7/R3b reliability prompt edits (slice-4)

authored and
jwbron
committed
06ac3ee
Persist BRC history for slice-4 (#2548)

egg-orchestrator
authored and
jwbron
committed
5bb941d
slice-4: changeset
jwbron
committed
a21b4ce
slice-4: drop BRC history artifacts from slice PR
jwbron
committed
fdf05d2
review: address slice-4 feedback (steering text, touched-lines scoping)
jwbron
committed
798016b
implement(documenter): R9 bounded-investigation instructions for reviewers (task-5-1)

authored and
jwbron
committed
238a355
implement(coder): R9 per-finding investigation tool-call cap (slice-5)

authored and
jwbron
committed
e5d1fd2
implement(tester): R9 per-finding tool-call cap tests (task-5-3)

authored and
jwbron
committed
8074db2
implement(tester): cover check() refusal previews (per-finding + run-total) — task-5-3

authored and
jwbron
committed
a01c756
slice-5: prettier fix + changeset
jwbron
committed
1340a42
review: address slice-5 feedback (wire the investigation cap to its callers)
jwbron
committed
7599eea
review: investigation-cap call sites run via npx tsx
jwbron
committed
3f01827
implement(documenter): slice-6 roster framework — always-on reviewers, model+effort defaults, kept gates

authored and
jwbron
committed
4e62371
Persist BRC history for slice-6 (#2548)

egg-orchestrator
authored and
jwbron
committed
8d21077
slice-6: changeset
jwbron
committed
481198d
slice-6: drop BRC history artifacts from slice PR
jwbron
committed
33de326
review: rework slice-6 roster to opt-in (no new default cost)
jwbron
committed
46228e0
review: genericize orchestrator (uniform findings contract, roster detail moved to sub-agents/README)
jwbron
committed
8f8cb7c
slice-7: eleven specialist lenses (rebased onto genericized orchestrator)
jwbron
committed
ad4bede
implement(coder): slice-9 — smoke benchmark (corpus + no-post runner + staged CI)

authored and
jwbron
committed
3e484a6
implement(tester): slice-9 smoke.test.ts — vitest CI gate over the smoke corpus (task-9-3)

authored and
jwbron
committed
f4160f2
Persist BRC history for slice-9 (#2548)

egg-orchestrator
authored and
jwbron
committed
6eb3bf8
slice-9: prettier fixes + changeset
jwbron
committed
54c36e2
slice-9: drop BRC history artifacts from slice PR
jwbron
committed
cc4b30b
review: drop planning identifiers from the smoke benchmark (slice-9)
jwbron
committed
53d0d9a
review: drop low-value smoke corpus size test
jwbron
committed
35216aa
implement(tester): slice-10 rebalance verification against smoke set (task-10-3)

authored and
jwbron
committed
1ab96d8
implement(documenter): slice-10 wave-2 rebalance — edits 8-13, refuter panel, R6 experiment

authored and
jwbron
committed
c4f33b7
Persist BRC history for slice-10 (#2548)

egg-orchestrator
authored and
jwbron
committed
f76c328
slice-10: prettier fix + changeset
jwbron
committed
23aba3b
slice-10: drop BRC history artifacts from slice PR
jwbron
committed
721f8a5
review: strip remaining plan identifiers (slice-10 and lens prompts)
jwbron
committed
fa0bfbb
review: drop the webapp-40536 experiment record (moves to the PR description)
jwbron
committed
38f3e75
review: drop the blocking-claim refuter panel; gate blocking on validator-confirmed claims
jwbron
committed
5d9fee4
review: strip a reintroduced plan identifier from the verdict-gate paragraph
jwbron
committed
766a8bc
implement(coder): slice-11 — full eval suite (datasets, metrics, judge, gates, version stamp)

authored and
jwbron
committed
1f735fa
implement(coder): slice-11 judge.ts — decouple from slice-8 type; harden aggregate join

authored and
jwbron
committed
cf342d2
implement(tester): slice-11 eval-suite self-tests + CI-gate guard (task-11-6)

authored and
jwbron
committed
992048c
Persist BRC history for slice-11 (#2548)

egg-orchestrator
authored and
jwbron
committed
14c38fb
slice-11: staged scheduled full-eval workflow with live judge (task-11-6, resolves gap-1)
jwbron
committed
b6db587
slice-11: prettier fixes + changeset
jwbron
committed
169a6b8
slice-11: drop BRC history artifacts from slice PR
jwbron
committed
96b0895
review: strip planning identifiers from the eval suite (slice-11)
jwbron
committed
4eafaf6
review: eval CI rework; deterministic suite rides node-ci, live judge is a committed script on a weekly workflow, version-stamp machinery dropped
jwbron
committed
af7ad3a

Commits on Jul 8, 2026

Merge remote-tracking branch 'origin/main' into egg/pipeline-dcdad92d/slice-11
jwbron
committed
cef4470