Skip to content

test(recovery): prove PostgreSQL restore invariants - #72

Merged
seonghobae merged 5 commits into
mainfrom
test/postgres-recovery-invariants-20260814
Aug 16, 2026
Merged

seonghobae merged 5 commits into
mainfrom
test/postgres-recovery-invariants-20260814

Conversation

@seonghobae

Copy link
Copy Markdown
Contributor

Why

ADR-0015, ADR-0017, the deployment/recovery architecture, and ROADMAP Phase 2 require real PostgreSQL recovery evidence before GA. Protected main already has a transactional migration-chain test, but it does not prove that persisted recovery-critical data can be streamed into a clean migrated schema without weakening provenance, tenant-scoped deduplication, or worker fencing state.

What

  • Add a real PostgreSQL 18 recovery acceptance test over the exact protected-main migration set.
  • Rebuild a clean target schema from repository migrations, then use PostgreSQL binary COPY streams to back up and restore representative durable state.
  • Verify immutable response provenance survives restore.
  • Verify tenant-scoped outbox identity still rejects a conflicting replay while allowing the same event reference in another tenant.
  • Verify in-flight inbox-consumption fencing token and lease expiry evidence survive restore.

This is intentionally not a production backup-service, deployed topology, RPO/RTO, or GA claim. It is a bounded restore-invariant fixture that closes one part of the accepted recovery evidence gap while production/profile-specific backup tooling and drills remain future evidence.

Verification intent

  • cargo test --test postgres_recovery_invariants
  • full Runtime CI, coverage/docstrings/security and independent review on the exact head

Base at branch creation: cc5850a0d1eacbbf16d03075534fce460a8286e6.

@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@seonghobae, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 71 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: ad413957-b647-4ddd-8baa-41ff94a599f8

📥 Commits

Reviewing files that changed from the base of the PR and between cc5850a and 7182384.

📒 Files selected for processing (1)
  • tests/postgres_recovery_invariants.rs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copy link
Copy Markdown
Contributor Author

@opencode-agent

Please perform a bounded independent current-head review of 4b8089c18219c85613d163fb0103c663f1fdfab5 against protected main cc5850a0d1eacbbf16d03075534fce460a8286e6. Exact-head Runtime CI, Security Scan, and SAST Semgrep are successful. Preserve the repository's recovery, coverage/docstring, and security gates; do not update the branch or merge.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor Author

@opencode-agent

Please perform a bounded independent current-head review of 7182384cda875243c1d684bee8adc2475b7b89f0 against protected main cc5850a0d1eacbbf16d03075534fce460a8286e6. Exact-head Runtime CI, Security Scan, SAST Semgrep, coverage-evidence, and the required OpenCode workflow check are successful. Review the PostgreSQL restore-invariant fixture for correctness, isolation, destructive-test safety, provenance/tenant/fencing preservation, and truthful recovery claims. Do not update the branch or merge. If acceptable, submit a formal non-author approval on this exact head so the live ruleset can evaluate it.

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head 7182384cda875243c1d684bee8adc2475b7b89f0.

  • Head SHA: 7182384cda875243c1d684bee8adc2475b7b89f0

  • Workflow run: 31866347137

  • Workflow attempt: 1

Coverage evidence

Coverage Decision

  • Result: FAIL
  • Test evidence: not proven passing
  • Docstring evidence: not proven passing when configured
  • Failure count: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Test: postgres_recovery_invariants.rs"]
  S1 --> I1["regression suite"]
  I1 --> R1["Review risk: Test: postgres_recovery_invariants.rs"]
  R1 --> V1["targeted test run"]
Loading

@opencode-agent

Copy link
Copy Markdown
Contributor

OpenCode Review Overview

  • Head SHA: 7182384cda875243c1d684bee8adc2475b7b89f0
  • Workflow run: 31866347137
  • Workflow attempt: 1
  • Gate result: REQUEST_CHANGES (approval step)

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head 7182384cda875243c1d684bee8adc2475b7b89f0.

  • Head SHA: 7182384cda875243c1d684bee8adc2475b7b89f0

  • Workflow run: 31866347137

  • Workflow attempt: 1

Coverage evidence

Coverage Decision

  • Result: FAIL
  • Test evidence: not proven passing
  • Docstring evidence: not proven passing when configured
  • Failure count: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Test: postgres_recovery_invariants.rs"]
  S1 --> I1["regression suite"]
  I1 --> R1["Review risk: Test: postgres_recovery_invariants.rs"]
  R1 --> V1["targeted test run"]
Loading

@seonghobae
seonghobae dismissed opencode-agent[bot]’s stale review August 16, 2026 14:05

Dismissed as obsolete blocker evidence after exact-head re-verification. This review's sole blocker is a failed coverage-evidence result from workflow run 31866347137 on unchanged head 7182384. The current exact same-head check suite has coverage-evidence=success and opencode-review=success, with Runtime CI, Security Scan, and SAST Semgrep successful and no failed, queued, in-progress, or null-conclusion exact-head checks. No inline review threads exist. This dismissal does not create an approval or weaken a current gate; it removes a formal REQUEST_CHANGES whose stated premise is contradicted by newer exact-head evidence.

@seonghobae
seonghobae merged commit 57663cd into main Aug 16, 2026
30 checks passed
cursor Bot pushed a commit that referenced this pull request Aug 16, 2026
The inherited #72 recovery fixture inserted a processing consumption
row without claim_deadline_at. Migration 0019 requires that column for
processing rows, and the deadline trigger is UPDATE-only, so exact-head
CI failed closed. Seed a valid persisted claim and assert the deadline
survives COPY restore.

Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
seonghobae added a commit that referenced this pull request Aug 16, 2026
#81 requires claim_deadline_at for processing consumption rows, and the
deadline trigger is UPDATE-only. The inherited #72 fixture omitted that
column, so exact-head CI failed closed on #96 after merging current main.
Seed a valid persisted claim and assert the deadline survives COPY restore.
cursor Bot pushed a commit that referenced this pull request Aug 16, 2026
The inherited #72 recovery fixture inserted a processing consumption
row without claim_deadline_at. Migration 0019 requires that column for
processing rows, and the deadline trigger is UPDATE-only, so exact-head
CI failed closed. Seed a valid persisted claim and assert the deadline
survives COPY restore.

Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
seonghobae added a commit that referenced this pull request Aug 16, 2026
Keep postgres_scoring_completion beside the landed result-snapshot
adapter. Restore Active PR #69 after the stale #76 heading, and seed the
#81 claim deadline on the inherited #72 recovery fixture so exact-head CI
can classify the processing restore row.
seonghobae added a commit that referenced this pull request Aug 16, 2026
Keep postgres_research_release beside the landed item-delivery adapter.
Restore Active PR #71 after the stale #76 heading, and seed the #81 claim
deadline on the inherited #72 recovery fixture so exact-head CI can
classify the processing restore row.
seonghobae added a commit that referenced this pull request Aug 16, 2026
Keep postgres_research_contribution beside the landed item-delivery
adapter. Restore Active PR #74 after the stale #76 heading, and seed the
#81 claim deadline on the inherited #72 recovery fixture so exact-head CI
can classify the processing restore row.
seonghobae added a commit that referenced this pull request Aug 16, 2026
#81 requires claim_deadline_at for processing consumption rows. The
inherited #72 fixture omitted that column after merge-of-main, so exact-
head CI cannot insert the restore row. Seed a valid persisted claim,
assert the deadline survives COPY restore, and restore Active PR #53.
seonghobae added a commit that referenced this pull request Aug 16, 2026
Keep Active PR #60 after the stale #76 heading, and seed the #81 claim
deadline on the inherited #72 recovery fixture so exact-head CI can
classify the processing restore row.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant