Skip to content

feat(release): rebuild reproducible evidence on current main - #145

Merged
seonghobae merged 18 commits into
mainfrom
feat/release-evidence-current-main
Aug 12, 2026
Merged

feat(release): rebuild reproducible evidence on current main#145
seonghobae merged 18 commits into
mainfrom
feat/release-evidence-current-main

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Controlled current-main replacement for #57

The original release-evidence PR #57 remained stacked on the now-obsolete tenant branch. Protected main has since integrated the tenant lifecycle replacement and substantial independent hardening. Fresh path comparison proved #57's unique semantic delta is 26 paths; only .github/workflows/ci.yml, AGENTS.md, ARCHITECTURE.md, CHANGELOG.md, and CLAUDE.md overlap protected-main movement.

This replacement starts from exact protected main 00ed6aabb82c1754f8b14fa85929cac56f68402b and replays the exact #57 result blobs for the other 21 non-overlapping release-evidence paths. The five overlapping current-main files are deliberately retained unchanged in the initial replay so exact-head CI and documentation/workflow contracts can identify the narrow composition actually required.

The preserved implementation includes deterministic dual clean builds, descriptor-relative/no-follow manifest publication, held-descriptor artifact identity checks, same-name inode/in-place mutation defenses, bounded enumeration and hashing, the release-acceptance workflow, packaging metadata, and focused security/reproducibility tests and doctoring.

No #57 checks, reviews, approvals, generated merge, or stale-base evidence transfers. No force-push, destructive rebase, conflict-side selection, gate weakening, package publication, release authority, or central dependency workaround is introduced.

Replacement proof

Merge boundary

Keep Draft until exact-head CI/security/coverage/package/reproducibility/release-acceptance and required central workflows are terminal-success, current-main overlap composition is resolved test-first, review threads are clear, and live rules are satisfied. Merge only on an unchanged exact head. Close #57 as superseded only after this replacement is coherent and protected-main integration is complete; then rebuild downstream #58 and successors in dependency order.

Summary by CodeRabbit

  • 새 기능

    • 동일한 소스와 고정된 빌드 환경에서 패키지를 반복 빌드하고 결과의 일치 여부를 검증합니다.
    • 릴리스 산출물의 파일 수, 메타데이터, 크기 및 SHA-256 해시를 포함한 검증 매니페스트를 생성합니다.
    • 릴리스 수용 결과를 보존된 아티팩트로 확인할 수 있습니다.
  • 개선 사항

    • 패키징 과정에서 라이선스 및 고지 파일이 포함됩니다.
    • 릴리스 검증과 파일 기록 과정의 안정성과 보안이 강화되었습니다.
  • 문서

    • 재현 가능한 릴리스 검증, 운영 절차 및 복구 방법을 문서화했습니다.

@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d03f0cf2-9b09-4746-a916-1f3f616f8f88

📥 Commits

Reviewing files that changed from the base of the PR and between dcc8b04 and 3db3c9a.

📒 Files selected for processing (8)
  • docs/adr/0003-reproducible-release-evidence.md
  • docs/doctoring/release-artifact-descriptor-verification.md
  • docs/doctoring/reproducible-release-evidence.md
  • docs/superpowers/plans/2026-08-06-release-evidence-dirfd-hardening.md
  • docs/superpowers/specs/2026-08-06-release-evidence-dirfd-hardening-design.md
  • tests/test_release_artifact_dirfd.py
  • tests/test_release_evidence_dirfd.py
  • tests/test_release_evidence_documentation.py
🚧 Files skipped from review as they are similar to previous changes (5)
  • docs/doctoring/release-artifact-descriptor-verification.md
  • docs/superpowers/plans/2026-08-06-release-evidence-dirfd-hardening.md
  • docs/superpowers/specs/2026-08-06-release-evidence-dirfd-hardening-design.md
  • tests/test_release_evidence_dirfd.py
  • docs/doctoring/reproducible-release-evidence.md

📝 Walkthrough

Walkthrough

릴리스 수용 워크플로와 uv_build 패키징을 추가했습니다. 두 번의 재현 가능한 빌드를 비교하고, descriptor-relative 검증과 원자적 매니페스트 기록으로 릴리스 증거를 생성합니다.

Changes

릴리스 증거

Layer / File(s) Summary
빌드 도구체인과 수용 워크플로
.github/workflows/release-acceptance.yml, pyproject.toml, Dockerfile, tests/test_*packaging*, tests/test_release_acceptance_workflow.py
고정된 Python·uv 환경에서 정확한 PR head를 두 번 빌드합니다. uv_build==0.12.1 패키징과 Docker 입력 파일을 검증합니다.
아티팩트 식별과 재현성 검증
pg_llm_batch/release_evidence.py, tests/test_release_artifact_dirfd.py, tests/test_release_artifact_identity_races.py, tests/test_release_evidence.py
wheel과 sdist의 이름, 유형, 크기, 파일 정체성 및 SHA-256을 검증합니다. 심볼릭 링크, 경로 변경, inode 교체 및 파일 변조를 거부합니다.
안전한 매니페스트 기록
pg_llm_batch/release_evidence.py, tests/test_release_evidence.py, tests/test_release_evidence_dirfd.py
매니페스트를 제한된 디스크립터 상대 경로에 기록합니다. 임시 파일과 fsync()를 사용하고 원자적으로 교체합니다. 실패 시 기존 파일을 보존합니다.
릴리스 보안 계약과 문서 검증
docs/adr/*, docs/doctoring/*, docs/superpowers/*, CHANGELOG.md, tests/*documentation.py
재현성, TOCTOU 방어, fail-closed 동작, 운영 절차, 롤백 및 권한 분리 계약을 문서화하고 테스트합니다.

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
  participant GitHubActions
  participant ExactHead
  participant UVBuild
  participant ReleaseEvidence
  participant ArtifactStore
  GitHubActions->>ExactHead: checkout exact PR head
  ExactHead->>UVBuild: build wheel and sdist in two clean trees
  UVBuild->>ReleaseEvidence: provide two artifact sets
  ReleaseEvidence->>ReleaseEvidence: verify identity, size, and SHA-256
  ReleaseEvidence->>ReleaseEvidence: write manifest atomically
  ReleaseEvidence->>ArtifactStore: retain manifest for 14 days
Loading

Possibly related issues

Possibly related PRs

Suggested labels: dependencies, docker

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 60.61% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed 제목은 현재 main 기준 재현 가능한 릴리스 증거 재생성이라는 주요 변경을 명확히 요약합니다.
Linked Issues check ✅ Passed 구현과 테스트는 [#57]의 재현 빌드, descriptor-relative 검증, 원자적 매니페스트 기록 및 권한 분리 요구를 충족합니다.
Out of Scope Changes check ✅ Passed 워크플로, 패키징, 보안 구현, 회귀 테스트와 관련 문서는 [#57]의 릴리스 증거 범위에 포함됩니다.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/release-evidence-current-main

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae
seonghobae marked this pull request as ready for review August 12, 2026 06:37

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 9

🧹 Nitpick comments (4)
tests/test_release_artifact_dirfd.py (2)

149-149: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Path.open 몽키패치는 사용되지 않습니다. 제거를 고려하십시오.

pg_llm_batch/release_evidence.py는 아티팩트를 오직 os.open(..., dir_fd=...)로만 엽니다. Path.open 경로는 호출되지 않으므로 racing_path_open은 절대 실행되지 않습니다. 이 스캐폴딩은 구현이 여전히 경로명 기반 열기를 사용한다는 잘못된 인상을 줍니다.

두 테스트에서 original_path_open, racing_path_open, 해당 monkeypatch.setattr(Path, "open", ...) 호출을 제거하면 의도가 명확해집니다.

Also applies to: 160-163, 176-176, 197-197, 207-210, 223-223

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/test_release_artifact_dirfd.py` at line 149, Remove the unused
Path.open monkeypatch scaffolding from both tests, including original_path_open,
racing_path_open, and each monkeypatch.setattr(Path, "open", ...) call. Keep the
existing os.open dir_fd race-testing logic and assertions unchanged.

328-333: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

raising=False를 제거하십시오.

_SECURE_ARTIFACT_FLAGS_AVAILABLE는 모듈에 실제로 존재합니다. raising=False는 향후 상수 이름이 바뀌어도 테스트가 조용히 통과하게 만듭니다. 기본값인 raising=True를 사용하면 이름 변경이 즉시 드러납니다. 같은 파일의 다른 테스트(tests/test_release_evidence_dirfd.py 75행)는 이미 기본값을 사용합니다.

♻️ 제안 수정
     monkeypatch.setattr(
         release_evidence,
         "_SECURE_ARTIFACT_FLAGS_AVAILABLE",
         False,
-        raising=False,
     )
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/test_release_artifact_dirfd.py` around lines 328 - 333, Remove the
explicit raising=False argument from the monkeypatch.setattr call targeting
release_evidence._SECURE_ARTIFACT_FLAGS_AVAILABLE, relying on the default
raising=True so renamed or missing constants cause the test to fail immediately.
tests/test_release_artifact_identity_races.py (1)

14-38: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

픽스처와 상수가 tests/test_release_artifact_dirfd.py와 중복됩니다.

DISTRIBUTION, VERSION, COMMIT, SOURCE_DATE_EPOCH, WHEEL, SDIST, _write_release, _verify가 두 파일에 거의 동일하게 존재합니다. 유일한 차이는 _write_releaseparents=True 여부입니다. 버전 또는 배포 이름이 바뀌면 두 파일을 함께 수정해야 합니다.

공유 conftest.py 또는 테스트 헬퍼 모듈로 이동하면 한 곳에서 관리됩니다. 이 변경은 선택 사항입니다.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/test_release_artifact_identity_races.py` around lines 14 - 38,
Optionally consolidate the duplicated release constants and helper functions
DISTRIBUTION, VERSION, COMMIT, SOURCE_DATE_EPOCH, WHEEL, SDIST, _write_release,
and _verify from the two test modules into a shared conftest.py or test helper
module. Reuse the shared definitions in both tests, preserving the only
behavioral difference: _write_release must retain its required
directory-creation semantics, including parents=True where needed.
pg_llm_batch/release_evidence.py (1)

114-140: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

두 개의 디스크립터 순회 함수가 거의 동일합니다. 공통 헬퍼로 통합하는 방법을 고려하십시오.

_open_release_directory_open_manifest_parent는 앵커 열기, 컴포넌트 반복, 이전 디스크립터 닫기, 실패 시 정리라는 동일한 구조를 반복합니다. 차이는 os.mkdir 생성 단계와 오류 메시지뿐입니다. 보안 순회 로직이 한 곳에만 존재하면 향후 한쪽만 수정되는 위험이 사라집니다.

예: _walk_directory_descriptor(anchor, parts, *, create_mode: int | None, missing_message: str) -> int 형태의 내부 헬퍼를 추가하고, 두 함수는 이를 호출하도록 변경하십시오.

이 변경은 선택 사항입니다. 현재 동작에는 문제가 없습니다.

Also applies to: 334-369

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pg_llm_batch/release_evidence.py` around lines 114 - 140, Optionally
consolidate the duplicated descriptor-walking logic from _open_release_directory
and _open_manifest_parent into a shared internal helper, such as
_walk_directory_descriptor, that opens the anchor, iterates components,
optionally creates missing directories, closes replaced descriptors, and cleans
up on failure. Preserve each caller’s existing creation behavior and error
messages while routing both through the common security traversal path.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/adr/0003-reproducible-release-evidence.md`:
- Around line 67-68: Update item 12 in the ADR to describe the implementation
order used by _write_manifest_payload: fsync the file, perform the
descriptor-relative os.rename() atomic replacement, then fsync the final parent
directory. Keep the existing payload-writing and synchronization details while
correcting only this ordering.

In `@docs/doctoring/release-artifact-descriptor-verification.md`:
- Around line 50-51: Update step 10 in the release artifact descriptor
verification document to require comparison of the complete initial entry
snapshots, not only the bounded name tuple. Describe that the snapshots include
device, inode, file type, size, mtime, and ctime, matching the behavior of
_scan_release_entries and _artifact_records so inode replacement is detected.

In `@docs/superpowers/plans/2026-08-06-release-evidence-dirfd-hardening.md`:
- Around line 24-26: Update
docs/superpowers/plans/2026-08-06-release-evidence-dirfd-hardening.md lines
24-26 to use protected commit 00ed6aabb82c1754f8b14fa85929cac56f68402b as the
stack base, and revise the PR numbers and dependency order at lines 125-126 from
.github#790 -> `#53` -> `#55` -> `#56` to match the current integrated state; update
docs/superpowers/specs/2026-08-06-release-evidence-dirfd-hardening-design.md
line 5 similarly so its Dependency entry references the current base commit and
PR information rather than PR `#55`.

In
`@docs/superpowers/specs/2026-08-06-release-evidence-dirfd-hardening-design.md`:
- Around line 34-36: Update item 5 to explicitly permit destinations that are
absent or existing regular files, while continuing to reject every existing
non-regular destination; keep the wording aligned with the behavior of
_validate_manifest_destination and the documented “absent or a regular file”
contract.

In `@pg_llm_batch/config.py`:
- Around line 106-112: Update the boolean conversion branch in the configuration
parser to avoid using bool(raw) for unrecognized non-empty strings. After
normalizing the input, treat invalid boolean values such as “maybe” or “false ”
according to the function docstring’s “fall back on error” contract, returning
the configured default or rejecting them during set, while preserving recognized
true and false values.
- Around line 204-206: Update the set and get methods to use the same
normalized, typed value for both database persistence and cache storage, so
string inputs such as false produce consistent typed results across instances.
Ensure dict and list values cannot share mutable references with callers by
returning defensive copies from get, or reject mutable values consistently.
Preserve the existing default fallback behavior.

In `@tests/test_release_artifact_dirfd.py`:
- Around line 233-256: Update racing_read in
test_verifier_refuses_in_place_mutation_during_streaming_hash to replace the
artifact contents with data whose length differs from the original, ensuring the
verifier detects the mutation through file-size metadata without relying on
timestamp precision.

In `@tests/test_release_evidence_dirfd.py`:
- Around line 250-264: Update the test around write_release_manifest to
monkeypatch and record os.close calls, then assert the descriptor captured by
failing_fdopen was closed. Remove the os.fstat-based assertion so validation
does not depend on descriptor-number reuse, while preserving the temporary-file
cleanup assertion.

In `@tests/test_release_evidence_documentation.py`:
- Around line 22-27: Update
test_release_evidence_documents_exact_build_toolchain to load uv.toml
required-version and assert the documentation uses uv 0.12.3, using the existing
tomllib/tomli fallback pattern for Python 3.10 compatibility. Update the
documented uv version expectations to 0.12.3 while preserving the existing
uv_build==0.12.1 and lockfile wording assertions.

---

Nitpick comments:
In `@pg_llm_batch/release_evidence.py`:
- Around line 114-140: Optionally consolidate the duplicated descriptor-walking
logic from _open_release_directory and _open_manifest_parent into a shared
internal helper, such as _walk_directory_descriptor, that opens the anchor,
iterates components, optionally creates missing directories, closes replaced
descriptors, and cleans up on failure. Preserve each caller’s existing creation
behavior and error messages while routing both through the common security
traversal path.

In `@tests/test_release_artifact_dirfd.py`:
- Line 149: Remove the unused Path.open monkeypatch scaffolding from both tests,
including original_path_open, racing_path_open, and each
monkeypatch.setattr(Path, "open", ...) call. Keep the existing os.open dir_fd
race-testing logic and assertions unchanged.
- Around line 328-333: Remove the explicit raising=False argument from the
monkeypatch.setattr call targeting
release_evidence._SECURE_ARTIFACT_FLAGS_AVAILABLE, relying on the default
raising=True so renamed or missing constants cause the test to fail immediately.

In `@tests/test_release_artifact_identity_races.py`:
- Around line 14-38: Optionally consolidate the duplicated release constants and
helper functions DISTRIBUTION, VERSION, COMMIT, SOURCE_DATE_EPOCH, WHEEL, SDIST,
_write_release, and _verify from the two test modules into a shared conftest.py
or test helper module. Reuse the shared definitions in both tests, preserving
the only behavioral difference: _write_release must retain its required
directory-creation semantics, including parents=True where needed.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 21f0af94-434d-4c32-ba45-4e31c9bd4550

📥 Commits

Reviewing files that changed from the base of the PR and between e0888d3 and dcc8b04.

📒 Files selected for processing (27)
  • .github/workflows/release-acceptance.yml
  • CHANGELOG.md
  • Dockerfile
  • docs/adr/0003-reproducible-release-evidence.md
  • docs/adr/0004-descriptor-pinned-release-artifact-verification.md
  • docs/doctoring/durable-lifecycle-failure-evidence.md
  • docs/doctoring/release-artifact-descriptor-verification.md
  • docs/doctoring/reproducible-release-evidence.md
  • docs/superpowers/plans/2026-08-06-release-evidence-dirfd-hardening.md
  • docs/superpowers/specs/2026-08-06-release-evidence-dirfd-hardening-design.md
  • pg_llm_batch/config.py
  • pg_llm_batch/durable_client.py
  • pg_llm_batch/release_evidence.py
  • pyproject.toml
  • tests/test_config_boolean_fallback.py
  • tests/test_config_collection_type_fallback.py
  • tests/test_container_packaging_contract.py
  • tests/test_lifecycle_failure_confidentiality.py
  • tests/test_packaging_metadata.py
  • tests/test_release_acceptance_workflow.py
  • tests/test_release_artifact_dirfd.py
  • tests/test_release_artifact_dirfd_documentation.py
  • tests/test_release_artifact_identity_races.py
  • tests/test_release_evidence.py
  • tests/test_release_evidence_dirfd.py
  • tests/test_release_evidence_dirfd_documentation.py
  • tests/test_release_evidence_documentation.py
💤 Files with no reviewable changes (4)
  • tests/test_config_boolean_fallback.py
  • docs/doctoring/durable-lifecycle-failure-evidence.md
  • tests/test_config_collection_type_fallback.py
  • tests/test_lifecycle_failure_confidentiality.py

Comment thread docs/adr/0003-reproducible-release-evidence.md Outdated
Comment thread docs/doctoring/release-artifact-descriptor-verification.md Outdated
Comment thread docs/superpowers/plans/2026-08-06-release-evidence-dirfd-hardening.md Outdated
Comment thread docs/superpowers/specs/2026-08-06-release-evidence-dirfd-hardening-design.md Outdated
Comment thread pg_llm_batch/config.py Outdated
Comment thread pg_llm_batch/config.py Outdated
Comment thread tests/test_release_artifact_dirfd.py
Comment thread tests/test_release_evidence_dirfd.py Outdated
Comment thread tests/test_release_evidence_documentation.py
@seonghobae
seonghobae marked this pull request as draft August 12, 2026 06:52
@seonghobae
seonghobae marked this pull request as ready for review August 12, 2026 09:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant