feat(migrations): add atomic checkpoint storage operator - #63
feat(migrations): add atomic checkpoint storage operator#63seonghobae wants to merge 30 commits into
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
|
@coderabbitai review Review exact source head |
seonghobae
left a comment
seonghobae
left a comment
There was a problem hiding this comment.
@opencode-agent @cwl-noema-review
Review-only request for exact source head c213fc549dbf024da43b459df29c0616cd741c96 against exact stacked base 2820aa36d8dedf7d89d1b745e5728acf3b913d2b. Independently verify bounded SQL reads, load-before-connect behavior, fixed transaction-level advisory-lock semantics, exact 0007→0008 order, one-commit/all-or-nothing behavior, live rollback and concurrency determinism, canonical body-free CLI evidence, exact-head CI checkout, standalone/MSA compatibility, documentation claims, and generated-artifact hygiene. Do not mutate the branch, create a repair workflow, mark ready, or merge. Treat the draft stack and missing qualifying independent approval as merge blockers.
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
base='2820aa36d8dedf7d89d1b745e5728acf3b913d2b'
head='c213fc549dbf024da43b459df29c0616cd741c96'
git cat-file -e "${base}^{commit}"
git cat-file -e "${head}^{commit}"
printf '%s\n' '=== merge-base ==='
git merge-base "$base" "$head"
printf '%s\n' '=== exact commits ==='
git show -s --format='%H%n%P%n%s' "$base"
git show -s --format='%H%n%P%n%s' "$head"
printf '%s\n' '=== changed files ==='
git diff --name-status "$base" "$head"
printf '%s\n' '=== changed file statistics ==='
git diff --stat "$base" "$head"
printf '%s\n' '=== candidate paths ==='
git diff --name-only "$base" "$head" |
rg -i '(checkpoint|migration|init|cli|workflow|action|readme|agents|claude|adr|architecture|operator|doctor|changelog|test|sql|toml|yaml|yml)$' || trueLength of output: 3575 🏁 Script executed: #!/bin/bash
set -euo pipefail
base='2820aa36d8dedf7d89d1b745e5728acf3b913d2b'
head='c213fc549dbf024da43b459df29c0616cd741c96'
printf '%s\n' '=== implementation outline ==='
ast-grep outline pg_llm_batch/checkpoint_migrations.py --items all
ast-grep outline pg_llm_batch/cli.py --items all
ast-grep outline pg_llm_batch/__init__.py --items all
printf '%s\n' '=== implementation and public API diff ==='
git diff --unified=80 "$base" "$head" -- \
pg_llm_batch/checkpoint_migrations.py \
pg_llm_batch/cli.py \
pg_llm_batch/__init__.py
printf '%s\n' '=== SQL and database helper candidates ==='
fd -t f -E .git | rg '(^|/)(0*00[78].*\.sql|.*migration.*\.sql|.*bootstrap.*\.py|.*database.*\.py|.*db.*\.py)$' || true
printf '%s\n' '=== direct test outlines ==='
ast-grep outline tests/test_checkpoint_migration_bounded_reads.py --items all
ast-grep outline tests/test_checkpoint_migration_operator.py --items all
ast-grep outline tests/test_checkpoint_migration_operator_integration.py --items all
printf '%s\n' '=== direct test diffs ==='
git diff --unified=45 "$base" "$head" -- \
tests/test_checkpoint_migration_bounded_reads.py \
tests/test_checkpoint_migration_operator.py \
tests/test_checkpoint_migration_operator_integration.py \
tests/test_bootstrap_cli.py
printf '%s\n' '=== CI diff ==='
git diff --unified=80 "$base" "$head" -- .github/workflows/ci.ymlLength of output: 58633 🏁 Script executed: #!/bin/bash
set -euo pipefail
base='2820aa36d8dedf7d89d1b745e5728acf3b913d2b'
head='c213fc549dbf024da43b459df29c0616cd741c96'
printf '%s\n' '=== canonical migrations ==='
cat -n pg_llm_batch/migrations/0007_result_stream_checkpoints.sql
cat -n pg_llm_batch/migrations/0008_result_checkpoint_audit_events.sql
printf '%s\n' '=== rollback migrations ==='
cat -n pg_llm_batch/migrations/rollback/0007_result_stream_checkpoints.sql
cat -n pg_llm_batch/migrations/rollback/0008_result_checkpoint_audit_events.sql
printf '%s\n' '=== CI and compatibility contract diffs ==='
git diff --unified=60 "$base" "$head" -- \
tests/test_checkpoint_audit_ci_contract.py \
tests/test_workflow_contracts.py \
tests/test_checkpoint_migration_operator_documentation.py
printf '%s\n' '=== documentation diffs ==='
git diff --unified=35 "$base" "$head" -- \
README.md AGENTS.md CLAUDE.md ARCHITECTURE.md CHANGELOG.md \
docs/adr/0010-atomic-checkpoint-schema-operator.md \
docs/checkpoint-storage-migrations.md \
docs/doctoring/checkpoint-migration-operator.md
printf '%s\n' '=== files outside the requested diff that define referenced contracts ==='
ast-grep outline pg_llm_batch/checkpoint_store.py --items all
ast-grep outline pg_llm_batch/checkpoint_audit.py --items all
ast-grep outline pg_llm_batch/db.py --items all
rg -n -C 3 'MIGRATION_PATH|AUDIT_MIGRATION_PATH|def apply_result_checkpoint_schema|def apply_result_checkpoint_audit_schema|def _require_psycopg' \
pg_llm_batch/checkpoint_store.py pg_llm_batch/checkpoint_audit.py pg_llm_batch/db.py
printf '%s\n' '=== generated-artifact and ignore hygiene ==='
git diff --check "$base" "$head"
git diff --name-only "$base" "$head" | rg -i '(^|/)(dist|build|__pycache__|\.pytest_cache|coverage|htmlcov|.*\.(pyc|pyo|whl|tar\.gz|zip|egg-info))($|/)' || true
git status --shortLength of output: 89577 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '=== declared psycopg dependency ==='
rg -n -C 2 'psycopg' pyproject.toml uv.lock
printf '%s\n' '=== transaction-context usage in repository ==='
rg -n -C 4 'with psycopg\.connect\(|\.commit\(\)' pg_llm_batch testsLength of output: 54709 🤖 web_search executed:
Length of output: 10862
I found no actionable issue in the requested scope. Verified items:
I did not execute repository code. This result is a static source review of the requested range.
|
|
Current exact-head/base audit after dependency-order ancestry reconciliation:
Older |
|
Closing as superseded by #80. Live revalidation shows #80 is open/mergeable at exact head |
1 participant
Commercial and acquisition gap
Durable checkpoint and accepted-save audit migrations exist, but existing PostgreSQL volumes had no package-owned operator command that validates both canonical files before database access, serializes concurrent invocations, applies migration 0007 before 0008 in one transaction, rolls both back on failure, and emits bounded machine-readable identity evidence.
Implemented bounded vertical slice
init-checkpoint-storageCLI and publicapply_checkpoint_schema_migrations()coordinator;init-db,apply_result_checkpoint_schema(), andapply_result_checkpoint_audit_schema()behavior;0007_result_stream_checkpointsand0008_result_checkpoint_audit_eventsbytes through immutable migration ID, positive bounded byte count, and lowercase SHA-256 descriptors;pg_advisory_xact_lock, executes migration 0007 before 0008 in one PostgreSQL transaction, and issues one explicit commit only after both statements succeed;${{ github.event.pull_request.head.sha || github.sha }}withpersist-credentials: falseand immediately verifygit rev-parse HEAD, preventing required evidence from silently running only on GitHub's synthetic merge ref;SHA-256 is documented only as deterministic byte-identification evidence, not a signature, authenticated provenance statement, remote attestation, publication authority, or release approval. Advisory locking coordinates cooperating package operators only; it is not authorization and does not constrain administrators or unrelated SQL clients.
No temporary or write-capable repair workflow, competing branch writer, generated coverage database, cache, build artifact, migration ledger, downgrade, destructive retained-evidence rollback, version bump, package publication, attestation, OIDC grant, or release is included.
Strict RED → GREEN → refactor evidence
2820aa36d8dedf7d89d1b745e5728acf3b913d2bonagent/checkpoint-audit-trail.9aedff7a50270a81cd245771dbe7f649a31fe66f; CI31149845743, Python 3.10 job92777048403, failed collection becauseCheckpointSchemaMigrationand the coordinator did not exist.c2de384dd5a42a026f1c49d9eeec92c5bff3217fpassed the new unit contract before subsequent test-first hardening. It is retained as development evidence only, not final merge evidence.049e6aab513494b114fddfe2e0679dc1ff19e921introduced the regression that forbidsPath.read_bytes()and negative-size reads. Its workflow was superseded before completion and is not counted as an executed pass or failure.6a7ac0125ea67ee60b88d41d3a6a9d56cf8cef75required both live integration suites. Its workflow was superseded before completion and is not counted as an executed pass or failure.2cb3607a7b9c696c58fa0df72743cd3667822685; CI31152186361, quality job92783956312, passed 423 tests then failed the new workflow contract because the required jobs still checked out the pull-request merge ref rather than the exact source head.25e8e493695c227e4510647305574025c0ec58eb; CI31152516949proved the exact-head checkout repair across all required jobs before authoritative doctoring was finalized.c213fc549dbf024da43b459df29c0616cd741c96.Current exact-head evidence
2820aa36d8dedf7d89d1b745e5728acf3b913d2b.31152717191: success on exact headc213fc549dbf024da43b459df29c0616cd741c96. Every required job's exact-source checkout assertion succeeded.92785563106: success with 658 passed and 8 deselected, Ruff success, 100% public docstrings, 100% production statement coverage, and 100% production branch coverage: 2,514 statements, 628 branches, 0 missed. Lock freshness and wheel/source-distribution build also succeeded.31152717233, job92785563352: success on the exact head, with exact-head checkout and reproducible wheel/source-distribution identity verification. This is acceptance evidence only and grants no versioning or publication authority.APPROVEDreview: absent.Dependency and merge boundary
Required order remains:
.github#790 -> pg-llm-batch#53 -> #55 -> #56 -> #57 -> #58 -> #59 -> #60 -> #61 -> #62 -> this PR.This PR remains intentionally draft. It must not be marked ready or merged until every prerequisite integrates into
main, the branch is reconciled onto the actual integrated base without losing predecessor fixes, and fresh integrated exact-head/exact-base quality, branch, docstring, security, dependency, migration, rollback, concurrency, packaging, container, provenance, supply-chain, release-acceptance, branch-protection, and independent-review gates succeed. Unresolved valid findings must be zero and a qualifying independent non-author GitHubAPPROVEDreview is mandatory. Stacked-base and synthetic merge-result evidence is not reusable as final integrated release evidence.