Skip to content

feat: seed PRs record their standards ref, supersede older seed PRs, and refuse downgrades (#1925) - #2256

Merged
don-petry merged 7 commits into
mainfrom
claude/issue-1925-seed-provenance-supersede
Oct 11, 2026
Merged

don-petry merged 7 commits into
mainfrom
claude/issue-1925-seed-provenance-supersede

Conversation

@don-petry

@don-petry don-petry commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

scripts/seed-repo-template.sh now says where a seed came from, keeps one seed PR open, and refuses to downgrade a template file.

  • Provenance. The seed PR body states the resolved standards ref, how it was chosen (channel, explicit STANDARDS_REF, HEAD fallback or local) and the commit. A hidden marker carries the same three values. A reused PR's body is rewritten on each run.
  • Supersede. When several seed-scaffold/* PRs are open, the newest is kept and each older one is closed with a comment linking it.
  • No downgrades. A file that differs from the run's emission but matches the emission at standards HEAD is refused, with "standards channel is behind the template — promote standards".
  • Operator procedure. docs/release/seeding-repo-template.md documents how and when to seed.

Closes #1925 ("seed-repo-template.sh: seed PRs record no standards ref, never supersede each other, and can regress repo-template").

Also in this PR

One sentence in AGENTS.md for issue #1927: seed-repo-template.sh has one home, this repo. The rename that removes the other same-named script is petry-projects/.github#1294, which closes #1927.

Builds on PR #2244

The no-downgrade check reuses the ancestry read, _template_drift_compare, that PR #2244 (issue #2237) added. That PR is merged, and this branch is up to date with main, so the diff here is this PR's four files only.

Problem

On 2026-09-23 a seed PR generated from standards/v1-stable (then v1.0.0) would have removed the merge_group trigger a previous seed had shipped. Five seed PRs were open at once, and none said which ref it used.

Acceptance criteria

  1. Provenance: done as described above.
  2. Supersede: issue seed-repo-template.sh strips the commercial-support sections from repo-template on every run — its README and BOOTSTRAP baseline predates the commercial-support standard, and stale seed PRs pile up #2233 landed while this was in progress and already makes a run update an open seed PR in place. This PR adds the missing part: the other open seed PRs are closed, each with a link to the kept one. A PR that cannot be closed fails the run and is named.
  3. No downgrades: done. If every differing file is refused, no PR is opened and the empty branch is removed.
  4. Scheduled home: a documented operator procedure, as chosen by the maintainer, not a workflow.

Failure handling

  • If the standards HEAD commit, its relation to the resolved ref, or the emission at HEAD cannot be read, the seed aborts. It never treats "cannot tell" as "not ahead".
  • If this run's PR number cannot be identified, no older seed PR is closed, and the run says so.
  • If the open pull requests cannot be listed, the seed stops before it writes anything. It could not rule out an open seed PR.
  • Every standards file is fetched at the commit recorded in the PR body, not through the ref name, so a channel that moves mid-run cannot make the two disagree.
  • Only a file that would be overwritten pays for the newer-commit comparison. With a local STANDARDS_DIR there is no ref, so the check is skipped.

Risk

Medium. This changes a script that writes to petry-projects/repo-template.

  • New writes: gh pr edit on the kept seed PR, gh pr close on older seed-scaffold/* PRs, and a ref DELETE of a branch this run created and left empty.
  • gh pr close only ever targets a PR whose head branch starts with seed-scaffold/.
  • A run that finds nothing to write used to fail at gh pr create. It now exits 0 with "no PR opened".

Test plan

  • bats tests/test_seed_repo_template.bats: 99 pass, 1 fails. The failure is test 59 (issue seed-repo-template.sh strips the commercial-support sections from repo-template on every run — its README and BOOTSTRAP baseline predates the commercial-support standard, and stale seed PRs pile up #2233's sed -i test), which fails on macOS sed on main too. 34 tests are new. Three of them run the real HEAD read, the real ancestry read and the real child emission with only gh replaced.
  • Counterfactual: with the seeder reverted to main, 18 of the new tests fail.
  • bats tests/template_stub_drift.bats tests/agent_ingress_reference.bats tests/test_lint_main_push_trigger.bats: all pass.
  • find scripts -name "*.sh" -exec shellcheck --severity=warning -x {} +: clean. markdownlint-cli2 on the new doc: 0 errors.
  • Run against live GitHub with real gh on 2026-10-11, DRY_RUN=true (read-only):
    • Default channel: every file SKIP or PRESERVE except README.md (UPDATE). Re-run after the commit pin was added: same result.
    • STANDARDS_REF=standards/v1.0.0: agent-shield.yml, dependency-audit.yml, copilot-setup-steps.yml, dependabot-rebase.yml and dev-lead.yml are reported would REFUSE … the template is ahead, with the channel-behind warning. That is the September regression, now refused.
  • Not run: a live (non-dry) seed. The PR create, edit, close and branch delete paths are covered with a stubbed gh only.

Rollback

Revert this PR. A seed PR opened by this version stays valid; its extra body text is harmless.

Monitoring

The next real seed of repo-template: its PR body should show the "Standards provenance" block, and only one seed-scaffold/* PR should be open afterwards.

Interaction contract

  • N/A — this PR does not add or change an agentic role.

Checklist

  • shellcheck scripts/*.sh passes for any changed shell scripts.
  • Docs updated where behavior or standards changed.

🤖 Generated with Claude Code

View guided diff Turn on auto-fix

don-petry and others added 3 commits October 11, 2026 00:59
…row misclassifications and three untested paths left open by PR 2219
…3' into claude/issue-1925-seed-provenance-supersede
…and refuse downgrades (#1925)

seed-repo-template.sh opened seed PRs that did not say which standards ref they
came from, left older seed PRs open, and could overwrite a template file with an
older version of itself.

- Provenance: the seed PR body states the resolved standards ref, how it was
  chosen and the commit. A reused PR's body is rewritten on each run.
- Supersede: #2233 already updates an open seed PR in place. When several are
  open the newest is kept and each older one is now closed with a comment
  linking it. A PR that cannot be closed fails the run.
- No downgrades: a file that differs from the run's emission but matches the
  emission at standards HEAD is refused, with "standards channel is behind the
  template — promote standards". If nothing else needs writing, no PR is opened
  and the empty branch is removed.
- A newer-commit check that cannot be made aborts the seed. It is never read as
  "not ahead".
- docs/release/seeding-repo-template.md is the operator procedure.

The no-downgrade check reuses the drift guard's HEAD and ancestry reads, so this
branch includes the unmerged #2244 (issue #2237) that adds the ancestry read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@don-petry
don-petry requested a review from a team as a code owner October 11, 2026 02:45
@don-petry don-petry added the dev-lead:hands-off Exclude from dev-lead agent automation label Oct 11, 2026
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@gemini-code-assist

Copy link
Copy Markdown

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@coderabbitai

coderabbitai Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 27 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 15782ec2-daf6-4a1b-89ee-13d96fd79573

📥 Commits

Reviewing files that changed from the base of the PR and between 82461dc and 70a1c74.


📒 Files selected for processing (4)
  • AGENTS.md
  • docs/release/seeding-repo-template.md
  • scripts/seed-repo-template.sh
  • tests/test_seed_repo_template.bats

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Comment thread scripts/seed-repo-template.sh Outdated
Comment thread scripts/seed-repo-template.sh
@codeant-ai

codeant-ai Bot commented Oct 11, 2026

Copy link
Copy Markdown

CodeAnt Nitpicks

4 code suggestions

1. STANDARDS_DIR bypasses the HEAD and ancestry checks, so content matching standards HEAD can be overwritten by a different local emission instead of refused.

Comment mismatch · AGENTS.md:533-534


2. When the channel ref is unavailable, _resolve_standards_ref falls back to default-branch HEAD, so leaving STANDARDS_REF unset does not guarantee a channel-based seed.

Comment mismatch · AGENTS.md:527


3. For STANDARDS_DIR, the PR body records unknown as the commit; this claim promises a commit although local-source provenance has none.

Comment mismatch · AGENTS.md:529


4. The body records _STD_SHA, but _fetch_standard fetches through mutable _STD_REF; if that ref moves during generation, the PR claims a commit different from its content.

Api mismatch · scripts/seed-repo-template.sh:1028

@codeant-ai

codeant-ai Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

CodeAnt PR Risk: Low Risk

  • The PR appears safe to merge; the prior concerns about missed seed PRs and stale dated branches are addressed in the current code.
  • The seeder pins reads to the recorded standards commit, refuses verified downgrades, and adds tests for these behaviors.

Assessed commit: 70a1c7445f98

@don-petry

Copy link
Copy Markdown
Collaborator Author

dev-lead is withholding action on this item.

It is labeled dev-lead:hands-off (deliberately kept off-limits to dev-lead (e.g. changes to the dev-lead automation itself)), so dev-lead will not pick it up while that label is present. This notice is posted once so the withhold is visible rather than looking like a stalled run.

To re-enable automated pickup: remove the dev-lead:hands-off label.

… open PR

- If today's seed branch already exists it may carry an earlier run's edits. It
  is now handled like an open seed PR's branch: every file is made to match what
  this run wants. Before, a file this run skipped, preserved or refused kept the
  earlier run's content, so a refused downgrade could still ship.
- The open-PR listing limit is applied before the seed-branch filter, so it is
  raised from 100 to 1000. A seed PR cut off by the limit would be missed and a
  second one opened.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@don-petry

Copy link
Copy Markdown
Collaborator Author

QA Lead — test-risk advisory

Risk tier: MEDIUM. The script can now edit and close PRs and delete branches on repo-template, but it fails closed, gh pr close only touches seed-scaffold/* branches, and the new tests are strong (22 new, 18 of which fail against main).

What I'd shore up (highest leverage first):

Escalate? No. The first live (non-dry) seed is the real integration test, so watch it as the PR's Monitoring section says.

Advisory only — I comment, I do not change code. Ground: BMAD Test Architecture
(bmad-tea v1.19.0). Opt out on this item with the qa-lead:hands-off label.

The standards repo's same-named script is a different tool and is being renamed
to emit-standard-file.sh in petry-projects/.github#1294.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@donpetry-bot

Copy link
Copy Markdown
Contributor

Advisory bots were rate-limited; auto-approval is withheld until they recover. pr-review-sweep will re-review this PR after 2026-10-11T04:14:10Z.

@donpetry-bot

Copy link
Copy Markdown
Contributor

Advisory bots were rate-limited; auto-approval is withheld until they recover. pr-review-sweep will re-review this PR after 2026-10-11T04:14:12Z.

Don Petry and others added 2 commits October 10, 2026 22:20
…d-provenance-supersede

# Conflicts:
#	AGENTS.md
…mit, and test the real ancestry path

- If the open pull requests cannot be listed, the seed stops before it writes
  anything. Before, a failed listing read as "no seed PR open", which opens a
  duplicate and leaves the older ones open.
- A seed run now fetches every standards file at the commit it records in the PR
  body, not through the ref name. A channel that moved mid-run could otherwise
  leave the PR naming a commit its content did not come from.
- A dated branch left by an earlier run, with nothing written and nothing the
  base lacks, opens no PR.
- Docs: a channel that cannot be resolved falls back to HEAD and the operator
  stops; a local STANDARDS_DIR has no commit and no downgrade check, so it is
  not for a real seed.
- Tests: three run the real HEAD read, the real ancestry read and the real
  child emission with only gh replaced. Others cover the unreadable PR list, a
  seed PR beyond the first 100 open PRs, the commit pin, a failed provenance
  update, and a PR number that cannot be read back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

@donpetry-bot

Copy link
Copy Markdown
Contributor

Advisory bots were rate-limited; auto-approval is withheld until they recover. pr-review-sweep will re-review this PR after 2026-10-11T15:43:51Z.

@don-petry
don-petry merged commit 13eea15 into main Oct 11, 2026
53 checks passed
@don-petry
don-petry deleted the claude/issue-1925-seed-provenance-supersede branch October 11, 2026 14:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dev-lead:hands-off Exclude from dev-lead agent automation

Projects

None yet

2 participants