Skip to content

fix(sync): converge delivery in stable reviewed PRs - #3049

Merged
stranske merged 7 commits into
mainfrom
codex/sync-stable-delivery-review
Aug 12, 2026
Merged

fix(sync): converge delivery in stable reviewed PRs#3049
stranske merged 7 commits into
mainfrom
codex/sync-stable-delivery-review

Conversation

@stranske

@stranske stranske commented Aug 11, 2026

Copy link
Copy Markdown
Owner

Summary

  • coalesce canary and promoted delivery into one stable PR per repository and lane
  • preserve review state for exact base/tree no-op refreshes and restore draft/staging holds before every real head mutation
  • use a capacity-aware reviewer policy: one response after the quiet period, bounded degradation for reviewer outages or no response, and no waiver for active review threads
  • seal the exact reviewed head, trigger a fresh required Gate, and block every shared merger while the staging hold remains
  • retain the strict behind-branch handling merged in fix(sync): update behind generated branches before merge #3047 and avoid closing a different generated-delivery lane as stale
  • update drift reporting, automation instructions, tests, templates, and operator docs for the stable delivery branch

Why

The previous hash-per-wave fan-out turned each source correction into another fleet-sized PR wave. It also had no durable reviewer-settled state, so recurring sync runs could restart the same review cycle and generic merge lanes could consume an in-place PR before delivery was complete.

This change makes PR identity stable while keeping plan and tree identity immutable in the delivery record. Review capacity never requires all configured reviewers, but unresolved live review debt remains a hard stop.

Validation

  • node --test .github/scripts/tests/*.test.js (1344 passed, 1 skipped)
  • python -m pytest -q tests/workflows tests/scripts/test_check_consumer_sync_drift.py (871 passed, 3 expected privilege skips)
  • focused post-rebase Python suite (142 passed)
  • actionlint on all changed workflows and consumer Gate template
  • Ruff on changed Python surfaces
  • validate_template_sync.py and validate_template_completeness.py
  • git diff --check

Rollout

This PR is intentionally draft during the mandatory post-push review window. After an unchanged-head review pass it can be marked ready; merging it starts one new candidate cycle, followed by Maint 71 candidate settlement/evidence and stable delivery promotion.

Summary by CodeRabbit

  • New Features
    • Introduced stable candidate and delivery workflows with staging, review, and sealed lifecycle states.
    • Added exact-head delivery sealing, reviewer settlement, lease validation, and capacity-aware review handling.
    • Added merge protections for unsealed or held delivery changes.
  • Documentation
    • Updated workflow, integration, operations, and maintenance guidance for stable delivery procedures.
  • Bug Fixes
    • Improved synchronization branch handling and prevented stale or mismatched delivery merges.
  • Tests
    • Expanded coverage for sealing, review eligibility, merge guards, branch stability, and delivery lifecycle behavior.

@cursor

cursor Bot commented Aug 11, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@coderabbitai

coderabbitai Bot commented Aug 11, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 29 minutes

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: f72fa433-7073-4f80-b4f3-d0577972a739

📥 Commits

Reviewing files that changed from the base of the PR and between d229f21 and 09420eb.

📒 Files selected for processing (21)
  • .github/scripts/__tests__/runtime-ac-merge-guard.test.js
  • .github/scripts/__tests__/sync_pr_lease_contract.test.js
  • .github/scripts/__tests__/sync_pr_merge_contract.test.js
  • .github/scripts/gate_summary.py
  • .github/scripts/maint71_merge_sync_prs.js
  • .github/scripts/runtime_ac_merge_guard.js
  • .github/scripts/sync_pr_lease_contract.js
  • .github/scripts/sync_pr_merge_contract.js
  • .github/sync-manifest.yml
  • .github/workflows/maint-68-sync-consumer-repos.yml
  • .github/workflows/maint-71-merge-sync-prs.yml
  • .github/workflows/pr-00-gate.yml
  • docs/SYNC_WORKFLOW.md
  • renovate-presets/consumer-managed-paths.json
  • templates/consumer-repo/.github/scripts/gate_summary.py
  • templates/consumer-repo/.github/scripts/runtime_ac_merge_guard.js
  • templates/consumer-repo/.github/scripts/sync_pr_lease_contract.js
  • templates/consumer-repo/.github/workflows/pr-00-gate.yml
  • tests/scripts/test_check_consumer_sync_drift.py
  • tests/workflows/github_scripts/test_gate_summary.py
  • tests/workflows/test_sync_manifest_delivery.py
📝 Walkthrough

Walkthrough

The PR replaces hash-based consumer delivery branches with stable candidate and delivery lanes. It adds staged delivery records, reviewer settlement, exact-head sealing, Gate validation, merge holds, workflow updates, tests, and operational documentation.

Changes

Stable delivery contracts and review

Layer / File(s) Summary
Delivery contracts and reviewer settlement
.github/scripts/sync_pr_lease_contract.js, .github/scripts/sync_pr_merge_contract.js, .github/scripts/__tests__/*, config/consumer_sync_review_policy.json
Delivery records now support staging, reviewing, and sealed states. Reviewer settlement handles quorum, quiet periods, capacity failures, and timeouts.
Stable branch production and delivery records
.github/workflows/maint-68-sync-consumer-repos.yml, scripts/check_consumer_sync_drift.py, tests/scripts/*, README.md, docs/*
Maint 68 uses stable branches, draft staging PRs, lifecycle metadata, and preserved review state. Documentation and drift reporting describe the stable delivery model.
Review, sealing, and merge execution
.github/scripts/maint71_merge_sync_prs.js, .github/workflows/maint-71-merge-sync-prs.yml
Maint 71 refreshes PR data, collects reviewer evidence, seals exact heads, requires fresh gates, retries merge checks, and updates labels after successful merges.
Gate seal validation and merge holds
.github/scripts/gate_summary.py, .github/scripts/runtime_ac_merge_guard.js, .github/workflows/pr-00-gate.yml, templates/consumer-repo/.github/*, tests/workflows/*
Gate validates sealed delivery records and exact head SHAs. Merge guards block staged deliveries unless explicitly allowed for sealed delivery processing.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related issues

Possibly related PRs

Suggested labels: verify:compare

Suggested reviewers: codex-automation

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 5.26% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: consolidating delivery into stable, reviewed pull requests.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/sync-stable-delivery-review

Comment @coderabbitai help to get the list of available commands.

Comment thread .github/workflows/pr-00-gate.yml Fixed
Comment thread templates/consumer-repo/.github/workflows/pr-00-gate.yml Fixed
Comment thread .github/scripts/sync_pr_merge_contract.js Fixed
Comment thread .github/workflows/pr-00-gate.yml Fixed
Comment thread .github/workflows/pr-00-gate.yml Fixed
@stranske
stranske marked this pull request as ready for review August 12, 2026 00:20
@stranske
stranske temporarily deployed to agent-high-privilege August 12, 2026 00:23 — with GitHub Actions Inactive

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d229f2107b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread templates/consumer-repo/.github/workflows/pr-00-gate.yml

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 12

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/scripts/__tests__/runtime-ac-merge-guard.test.js:
- Around line 63-83: Add a test case in the mutable generated delivery coverage
that supplies both sync:delivery-staging and a runtime AC label while setting
allowSealedSyncDelivery: true, then assert the runtime AC requirement still
rejects the request. Keep the existing generic-merger rejection and sealed Maint
71 success assertions unchanged.

In @.github/scripts/__tests__/sync_pr_merge_contract.test.js:
- Around line 363-393: Extend the test named “reviewer settlement never requires
every configured reviewer” with an assertion for the interval after the quiet
period but before maxWaitMs, using no responses and available reviewers. Verify
evaluateReviewerSettlement returns ready: false, reason:
'review_quorum_pending', and the expected eligible_at value, covering the path
later mapped to reviewer_settlement_pending.

In @.github/scripts/maint71_merge_sync_prs.js:
- Around line 278-310: Wrap the GitHub GraphQL request in
collectReviewerEvidence with the existing withRetry pattern and a try/catch,
matching activeReviewThreadCount’s failure handling. On any exhausted or caught
API failure, warn and return empty responded/unavailable lists with truncated:
true so evaluateReviewerSettlement remains pending and cannot reach quorum from
missing evidence.
- Around line 844-846: Update the head timestamp assignment near
evaluatePostPushReviewWindow so it uses a server-observed push anchor: retain
the existing pushed timestamp when available, or clamp the selected commit date
to at least pr.head.updated_at. Do not rely solely on
selectedHeadCommit.comitter.date or author.date, ensuring backdated commits
cannot make the post-push review window immediately eligible.

In @.github/scripts/sync_pr_lease_contract.js:
- Around line 95-101: Update replaceDeliveryRecord to pass a replacer function
to String.prototype.replace instead of using marker directly as the replacement
string. Preserve the existing expression matching and return marker unchanged
from the callback so `$&`, `$'`, `$```, and `$<name>` sequences in serialized
delivery-record fields cannot be interpreted.

In @.github/scripts/sync_pr_merge_contract.js:
- Around line 209-256: In .github/scripts/sync_pr_merge_contract.js lines
209-256, update the reviewer settlement logic around the contract function to
normalize quietPeriodMs, maxWaitMs, and minimumResponses to finite defaults
before date arithmetic and eligible_at generation. In
.github/scripts/maint71_merge_sync_prs.js lines 182-194, validate the parsed
policy during load, rejecting or defaulting non-finite minimum_responses,
quiet_period_minutes, and maximum_wait_minutes, and wrap fs.readFileSync plus
JSON.parse to report clear failures for missing or malformed policy files before
the repository loop starts.

In @.github/workflows/maint-68-sync-consumer-repos.yml:
- Around line 924-930: Update the exact no-op handling around the existing PR
matching logic to detect valid legacy records lacking delivery_state, write the
staging lifecycle record as a metadata-only migration, and avoid any commit or
push. Preserve the current review lifecycle and early-exit behavior for already
initialized records, and add coverage for a valid legacy record whose base and
tree are unchanged.
- Line 779: Update both gh pr list lookups at the existing branch-name queries,
including the one assigned to existing_pr, to restrict results to
same-repository pull requests by filtering isCrossRepository == false or
matching the target headRepository identity. Preserve selection of the intended
local branch PR, and add a regression case covering a fork PR named
sync/workflows-delivery.

In @.github/workflows/maint-71-merge-sync-prs.yml:
- Line 183: Update the merge step’s always() condition to also require
successful selector resolution and repository extraction, and ensure an empty
ACTIVE_SYNC_HASH_INPUT cannot permit unscoped processing. Preserve the existing
candidateEvidenceAllowsMutation checks while preventing merges for any
repository when either prerequisite fails.

In @.github/workflows/pr-00-gate.yml:
- Around line 165-212: Require the PR head repository to equal github.repository
in both delivery-seal step conditions in
.github/workflows/pr-00-gate.yml:165-212 and
templates/consumer-repo/.github/workflows/pr-00-gate.yml:174-221. In
.github/scripts/gate_summary.py:55-100 and
templates/consumer-repo/.github/scripts/gate_summary.py:55-100, make stable
branches from forked repositories return (False, True, ""). Extend
tests/workflows/github_scripts/test_gate_summary.py:250-307 with a forked
stable-branch event and assert it is not classified as a generated delivery.

In `@docs/SYNC_WORKFLOW.md`:
- Around line 111-112: Update the Summary Checklist near the legacy merge
guidance to remove instructions for closing duplicate sync PRs and merging clean
PRs. Replace those items with Maint 71 reconciliation steps that preserve the
stable-branch quiet-period and recorded check-timestamp workflow.

In `@scripts/check_consumer_sync_drift.py`:
- Line 661: Add an assertion in
test_build_report_marks_current_sync_pr_as_covered that verifies
sync_remediation.delivery_branch matches the expected delivery branch value,
alongside the existing expected_branch assertion. Keep the test focused on the
new report contract.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 85973cac-dabf-45ef-88bb-5e5bbcf0f6e9

📥 Commits

Reviewing files that changed from the base of the PR and between d944013 and d229f21.

📒 Files selected for processing (28)
  • .github/scripts/__tests__/runtime-ac-merge-guard.test.js
  • .github/scripts/__tests__/sync_pr_lease_contract.test.js
  • .github/scripts/__tests__/sync_pr_merge_contract.test.js
  • .github/scripts/gate_summary.py
  • .github/scripts/maint71_merge_sync_prs.js
  • .github/scripts/runtime_ac_merge_guard.js
  • .github/scripts/sync_pr_lease_contract.js
  • .github/scripts/sync_pr_merge_contract.js
  • .github/workflows/maint-68-sync-consumer-repos.yml
  • .github/workflows/maint-71-merge-sync-prs.yml
  • .github/workflows/pr-00-gate.yml
  • README.md
  • config/consumer_sync_review_policy.json
  • docs/INTEGRATION_GUIDE.md
  • docs/SYNC_WORKFLOW.md
  • docs/WORKFLOW_GUIDE.md
  • docs/ci/WORKFLOWS.md
  • docs/ci/WORKFLOW_SYSTEM.md
  • docs/ops/CONSUMER_REPO_MAINTENANCE.md
  • docs/ops/DURABLE_TRACKING_ISSUES.md
  • docs/ops/SYNC_DEPENDENCY_CAMPAIGN.md
  • scripts/check_consumer_sync_drift.py
  • templates/consumer-repo/.github/scripts/gate_summary.py
  • templates/consumer-repo/.github/scripts/runtime_ac_merge_guard.js
  • templates/consumer-repo/.github/workflows/pr-00-gate.yml
  • tests/scripts/test_check_consumer_sync_drift.py
  • tests/workflows/github_scripts/test_gate_summary.py
  • tests/workflows/test_sync_manifest_delivery.py

Comment thread .github/scripts/__tests__/runtime-ac-merge-guard.test.js
Comment thread .github/scripts/__tests__/sync_pr_merge_contract.test.js
Comment thread .github/scripts/maint71_merge_sync_prs.js Outdated
Comment thread .github/scripts/maint71_merge_sync_prs.js
Comment thread .github/scripts/sync_pr_lease_contract.js
Comment thread .github/workflows/maint-68-sync-consumer-repos.yml Outdated
Comment thread .github/workflows/maint-71-merge-sync-prs.yml Outdated
Comment thread .github/workflows/pr-00-gate.yml
Comment thread docs/SYNC_WORKFLOW.md
Comment thread scripts/check_consumer_sync_drift.py
@agents-workflows-bot

agents-workflows-bot Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Automated Status Summary

Head SHA: 5e4e627
Latest Runs: ⏳ pending — Gate
Required contexts: summary
Required: core tests (3.12): ⏳ pending, core tests (3.13): ⏳ pending, docker smoke: ⏳ pending, gate: ⏳ pending

Workflow / Job Result Logs
(no jobs reported) ⏳ pending

Coverage Overview

  • Coverage history entries: 1

Coverage Trend

Metric Value
Current 76.26%
Baseline 85.00%
Delta -8.74%
Minimum 70.00%
Status ✅ Pass

Top Coverage Hotspots (lowest coverage)

File Coverage Missing
scripts/issue_dedup_smoke.py 0.0% 4
scripts/runner_lib/__main__.py 0.0% 3
scripts/validate_template_sync.py 0.0% 81
scripts/langchain/topic_splitter.py 19.1% 57
tools/codex_log_analyzer.py 19.6% 140
scripts/repo_review_round2_runner.py 25.1% 344
scripts/prune_agent_stubs.py 39.7% 26
scripts/repo_review_round1_runner.py 40.7% 133
tools/ensure_workflow_timeout_variables.py 42.1% 74
scripts/sync_label_docs.py 42.9% 64
tools/discover_model_catalog.py 44.8% 55
scripts/repo_review_backlog_scan.py 45.3% 116
scripts/repo_review_body_writer.py 46.5% 86
tools/codex_session_analyzer.py 47.9% 59
scripts/create_verifier_labels.py 48.3% 58

Low Coverage Files (<50.0%)

File Coverage Missing
scripts/issue_dedup_smoke.py 0.0% 4
scripts/runner_lib/__main__.py 0.0% 3
scripts/validate_template_sync.py 0.0% 81
scripts/langchain/topic_splitter.py 19.1% 57
tools/codex_log_analyzer.py 19.6% 140
scripts/repo_review_round2_runner.py 25.1% 344
scripts/prune_agent_stubs.py 39.7% 26
scripts/repo_review_round1_runner.py 40.7% 133
tools/ensure_workflow_timeout_variables.py 42.1% 74
scripts/sync_label_docs.py 42.9% 64
tools/discover_model_catalog.py 44.8% 55
scripts/repo_review_backlog_scan.py 45.3% 116
scripts/repo_review_body_writer.py 46.5% 86
tools/codex_session_analyzer.py 47.9% 59
scripts/create_verifier_labels.py 48.3% 58

Updated automatically; will refresh on subsequent CI/Docker completions.


Keepalive checklist

Scope

No scope information available

Tasks

  • No tasks defined

Acceptance criteria

  • No acceptance criteria defined

@stranske
stranske temporarily deployed to agent-high-privilege August 12, 2026 00:43 — with GitHub Actions Inactive
@stranske
stranske temporarily deployed to agent-high-privilege August 12, 2026 00:45 — with GitHub Actions Inactive
@stranske
stranske temporarily deployed to agent-high-privilege August 12, 2026 00:51 — with GitHub Actions Inactive
@stranske
stranske merged commit 5e4e627 into main Aug 12, 2026
57 checks passed
@stranske
stranske deleted the codex/sync-stable-delivery-review branch August 12, 2026 00:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex codex-automation verify:compare Compare multiple LLM evaluations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants