From daf56f859f2fe3a5ef892487d4867b23fcf7b14e Mon Sep 17 00:00:00 2001 From: James Wiesebron Date: Thu, 9 Jul 2026 13:54:06 -0700 Subject: [PATCH] [jwbron/live-ab-acceptance-control] review: [do-not-merge] live A/B acceptance control A comment-only frontmatter edit (one period) that no model-facing prompt contains: extracted agent prompts are byte-identical across arms, so the A/B report's deltas measure pure run-to-run model variance. That measured noise floor is the acceptance criterion for the control: near-zero deltas and the identical-arms note. --- .changeset/live-ab-acceptance-control.md | 4 ++++ workflows/review/review.md | 2 +- 2 files changed, 5 insertions(+), 1 deletion(-) create mode 100644 .changeset/live-ab-acceptance-control.md diff --git a/.changeset/live-ab-acceptance-control.md b/.changeset/live-ab-acceptance-control.md new file mode 100644 index 00000000..a68eee6e --- /dev/null +++ b/.changeset/live-ab-acceptance-control.md @@ -0,0 +1,4 @@ +--- +--- + +Do-not-merge acceptance control for the live A/B workflow: a comment-only review.md edit expecting near-zero arm deltas. diff --git a/workflows/review/review.md b/workflows/review/review.md index 678b9c04..78daf87b 100644 --- a/workflows/review/review.md +++ b/workflows/review/review.md @@ -22,7 +22,7 @@ on: # forks and automated branches are excluded by the `if:` condition below. roles: all -# Skip automated deploy PRs (`deploy/*`) and the changeset release PR — branch conventions +# Skip automated deploy PRs (`deploy/*`) and the changeset release PR. — branch conventions # shared across the repos this workflow runs in. Everything else is reviewed, including # pushes from our bots (`khan-actions-bot` and `github-actions[bot]`), since even automated # commits can carry real code changes worth reviewing.