Skip to content

feat(release): prove reproducible artifact identity - #55

Closed
seonghobae wants to merge 62 commits into
agent/tenant-scoped-lifecyclefrom
agent/reproducible-release-acceptance
Closed

feat(release): prove reproducible artifact identity#55
seonghobae wants to merge 62 commits into
agent/tenant-scoped-lifecyclefrom
agent/reproducible-release-acceptance

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Superseded by #57

Closed unmerged after live ancestry and security-boundary review proved that the unchanged final release-evidence head in #57 already contains this PR in full and should be reviewed as the only protected-main release slice.

Exact consolidation evidence:

All source history, RED/GREEN evidence, and prior CI/Release Acceptance runs remain available on this branch as development provenance. None transfers as final acceptance to #57 after its base change. #57 must reacquire all required exact-head/exact-base gates and independent approval before merge.

@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 044657b2-0d77-43c0-b9d3-a4db20a52623

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@seonghobae seonghobae changed the title test(release): define reproducible artifact acceptance feat(release): prove reproducible artifact identity Aug 5, 2026

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review exact current draft head 99ecff7aa98f8273490814d77b60e08ad799c958 against exact stacked base 27a2e6386e956986405ae6c7fb6ed9f615a433ae. Earlier requests and analyses are stale. Exact-head Release Acceptance run 31066416309 and current GitHub-generated test-merge CI run 31066416323 are complete and successful. Review the at-most-three-entry artifact scan, direct and nested manifest-parent symlink refusal before directory creation, destination and temporary-path handling, bounded hashing and diagnostics, exact uv/uv_build 0.12.1 controls, clean-source double builds, immutable action pins, read-only permissions, legal metadata, and stacked-base evidence boundaries. Report only exact-head actionable findings; do not treat draft-stage absence of default-branch required workflows as success or submit approval for predecessor evidence.

@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

repo='ContextualWisdomLab/pg-llm-batch'
head_sha='99ecff7aa98f8273490814d77b60e08ad799c958'
base_sha='27a2e6386e956986405ae6c7fb6ed9f615a433ae'
tmp_dir="$(mktemp -d)"
trap 'rm -rf "$tmp_dir"' EXIT

echo '== Commit identity and ancestry =='
git rev-parse "${head_sha}^{commit}"
git rev-parse "${base_sha}^{commit}"
git show -s --format='head=%H%nparents=%P%nsubject=%s' "$head_sha"
git merge-base --is-ancestor "$base_sha" "$head_sha" && echo 'base_is_ancestor=true' || echo 'base_is_ancestor=false'

echo '== Changed paths and whitespace =='
git diff --name-status "$base_sha" "$head_sha"
git diff --check "$base_sha" "$head_sha"

echo '== Exact-head implementation outline =='
git show "${head_sha}:pg_llm_batch/release_evidence.py" > "$tmp_dir/release_evidence.py"
ast-grep outline "$tmp_dir/release_evidence.py" --items all --type function,class

echo '== Exact-head verifier, workflows, packaging, and tests =='
for path in \
  'pg_llm_batch/release_evidence.py' \
  '.github/workflows/release-acceptance.yml' \
  '.github/workflows/ci.yml' \
  'pyproject.toml' \
  'Dockerfile' \
  'tests/test_release_evidence.py' \
  'tests/test_release_acceptance_workflow.py' \
  'tests/test_container_packaging_contract.py' \
  'tests/test_packaging_metadata.py' \
  'tests/test_release_evidence_documentation.py' \
  'docs/doctoring/reproducible-release-evidence.md' \
  'docs/adr/0003-reproducible-release-evidence.md'
do
  printf '\n===== %s =====\n' "$path"
  git show "${head_sha}:${path}" | nl -ba
done

echo '== Focused exact base-to-head diff =='
git diff --no-ext-diff --unified=45 "$base_sha" "$head_sha" -- \
  'pg_llm_batch/release_evidence.py' \
  '.github/workflows/release-acceptance.yml' \
  '.github/workflows/ci.yml' \
  'pyproject.toml' 'uv.lock' 'Dockerfile' \
  'tests/test_release_evidence.py' \
  'tests/test_release_acceptance_workflow.py' \
  'tests/test_container_packaging_contract.py' \
  'tests/test_packaging_metadata.py' \
  'tests/test_release_evidence_documentation.py' \
  'docs/doctoring/reproducible-release-evidence.md' \
  'docs/adr/0003-reproducible-release-evidence.md'

echo '== Exact-head control references =='
git grep -nE \
  'uv==0\.12\.1|uv_build==0\.12\.1|SOURCE_DATE_EPOCH|PYTHONHASHSEED|TZ=|LC_ALL=|git archive|persist-credentials|permissions:|release-manifest|sha256|symlink|is_file|is_symlink|islice|SLSA v1\.2|stacked|stale|retarget' \
  "$head_sha" -- \
  '.github' 'pyproject.toml' 'Dockerfile' 'pg_llm_batch' 'tests' 'docs' || true

echo '== Exact workflow run provenance =='
for run_id in 31066416309 31066416323; do
  gh api "repos/${repo}/actions/runs/${run_id}" \
    --jq '{id,name,head_sha,event,status,conclusion,workflow_id,html_url,created_at,updated_at}'
done

echo '== Current pull request state, informational only =='
gh pr view 55 --repo "$repo" \
  --json headRefOid,baseRefOid,isDraft,mergeCommit,statusCheckRollup \
  --jq '{headRefOid,baseRefOid,isDraft,mergeCommit,statusCheckRollup:[.statusCheckRollup[]|{name,status,conclusion,headSha}]}'

Length of output: 4900


🏁 Script executed:

#!/bin/bash
set -euo pipefail

repo='ContextualWisdomLab/pg-llm-batch'
head_sha='99ecff7aa98f8273490814d77b60e08ad799c958'
base_sha='27a2e6386e956986405ae6c7fb6ed9f615a433ae'

print_file() {
  path="$1"
  printf '\n===== %s =====\n' "$path"
  git show "${head_sha}:${path}" | awk '{printf "%5d  %s\n", NR, $0}'
}

echo '== Exact-head verifier, workflows, package configuration, and tests =='
for path in \
  'pg_llm_batch/release_evidence.py' \
  '.github/workflows/release-acceptance.yml' \
  '.github/workflows/ci.yml' \
  'pyproject.toml' \
  'Dockerfile' \
  'tests/test_release_evidence.py' \
  'tests/test_release_acceptance_workflow.py' \
  'tests/test_container_packaging_contract.py' \
  'tests/test_packaging_metadata.py' \
  'tests/test_release_evidence_documentation.py'
do
  print_file "$path"
done

echo '== Exact-head documentation excerpts =='
for path in \
  'docs/doctoring/reproducible-release-evidence.md' \
  'docs/adr/0003-reproducible-release-evidence.md'
do
  printf '\n===== %s =====\n' "$path"
  git show "${head_sha}:${path}" |
    grep -n -E -C 4 'stack|base|merge|retarget|head|stale|SLSA|artifact|manifest|uv|SOURCE_DATE_EPOCH|permission|pin|symlink' || true
done

echo '== Focused exact base-to-head diff =='
git diff --no-ext-diff --unified=45 "$base_sha" "$head_sha" -- \
  'pg_llm_batch/release_evidence.py' \
  '.github/workflows/release-acceptance.yml' \
  '.github/workflows/ci.yml' \
  'pyproject.toml' 'uv.lock' 'Dockerfile' \
  'tests/test_release_evidence.py' \
  'tests/test_release_acceptance_workflow.py' \
  'tests/test_container_packaging_contract.py' \
  'tests/test_packaging_metadata.py' \
  'tests/test_release_evidence_documentation.py' \
  'docs/doctoring/reproducible-release-evidence.md' \
  'docs/adr/0003-reproducible-release-evidence.md'

echo '== Exact-head control references =='
git grep -nE \
  'uv==0\.12\.1|uv_build==0\.12\.1|SOURCE_DATE_EPOCH|PYTHONHASHSEED|TZ=|LC_ALL=|git archive|persist-credentials|permissions:|release-manifest|sha256|symlink|is_file|is_symlink|islice|SLSA v1\.2|stacked|stale|retarget' \
  "$head_sha" -- \
  '.github' 'pyproject.toml' 'Dockerfile' 'pg_llm_batch' 'tests' 'docs' || true

echo '== Exact workflow run provenance =='
for run_id in 31066416309 31066416323; do
  gh api "repos/${repo}/actions/runs/${run_id}" \
    --jq '{id,name,head_sha,event,status,conclusion,workflow_id,html_url,created_at,updated_at}'
done

echo '== Current pull request state, informational only =='
gh pr view 55 --repo "$repo" \
  --json headRefOid,baseRefOid,isDraft,mergeCommit,statusCheckRollup \
  --jq '{headRefOid,baseRefOid,isDraft,mergeCommit,statusCheckRollup:[.statusCheckRollup[]|{name,status,conclusion,headSha}]}'

Length of output: 139160


@seonghobae I found one actionable finding on exact head 99ecff7aa98f8273490814d77b60e08ad799c958.

  • Line 69 — Extra-artifact diagnostics remain non-deterministic. _release_paths() takes the first three entries from Path.iterdir() and then sorts that selected subset. Path.iterdir() does not guarantee an order. If the directory has more than three entries, separate runs can select different entry names. The verifier correctly fails closed, but entries=[...] in the error at Lines 71-74 can differ between runs. Use a fixed count-failure message for the three-entry case, or omit names from that diagnostic.

I verified that the supplied base is an ancestor of the supplied head.

I verified that Release Acceptance run 31066416309 and CI run 31066416323 completed successfully for this exact head.

I also verified the direct and nested manifest-parent symlink checks before mkdir(), destination and temporary-path rejection, 1 MiB bounded SHA-256 reads, canonical atomic manifest output, clean git archive double builds, exact uv 0.12.1 and uv_build==0.12.1 pins, immutable action pins, read-only permissions, and legal metadata inputs.

The PR is a draft and has no GitHub-generated merge commit in the current PR state. I did not treat this stacked-base evidence or absent default-branch required workflows as final merge evidence. I did not submit approval.

You are interacting with an AI system.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review exact current draft head 3660bb9edd6351a9c02d9507f08ed647ddbf0d3a against exact stacked base 27a2e6386e956986405ae6c7fb6ed9f615a433ae. All earlier review requests and predecessor analyses are stale. Exact-head Release Acceptance run 31067011861 and current GitHub-generated test-merge CI run 31067011832 are complete and successful. Verify that the valid prior finding is fully addressed: missing or extra artifact counts now emit one fixed filesystem-order-independent diagnostic, while the verifier still scans at most three entries and may expose only sorted bounded names for exactly-two wrong-kind entries. Also review parent-chain symlink refusal before directory creation, destination and temporary-path handling, bounded hashing, exact uv/uv_build 0.12.1 controls, clean-source double builds, immutable action pins, read-only permissions, legal metadata, and stacked-base evidence boundaries. Report only exact-head actionable findings; do not treat draft-stage absence of default-branch required workflows as success or reuse predecessor-head evidence.

@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

repo='ContextualWisdomLab/pg-llm-batch'
head_sha='3660bb9edd6351a9c02d9507f08ed647ddbf0d3a'
base_sha='27a2e6386e956986405ae6c7fb6ed9f615a433ae'
tmp_dir="$(mktemp -d)"
trap 'rm -rf "$tmp_dir"' EXIT

echo '== Commit identity and ancestry =='
git rev-parse "${head_sha}^{commit}"
git rev-parse "${base_sha}^{commit}"
git show -s --format='head=%H%nparents=%P%nsubject=%s' "$head_sha"
git merge-base --is-ancestor "$base_sha" "$head_sha" && echo 'base_is_ancestor=true' || echo 'base_is_ancestor=false'

echo '== Base-to-head changed paths and whitespace =='
git diff --name-status "$base_sha" "$head_sha"
git diff --check "$base_sha" "$head_sha"

echo '== Python implementation outline =='
git show "${head_sha}:pg_llm_batch/release_evidence.py" > "$tmp_dir/release_evidence.py"
ast-grep outline "$tmp_dir/release_evidence.py" --items all --type function,class

echo '== Exact-head implementation, workflows, configuration, and tests =='
for path in \
  'pg_llm_batch/release_evidence.py' \
  '.github/workflows/release-acceptance.yml' \
  '.github/workflows/ci.yml' \
  'pyproject.toml' \
  'Dockerfile' \
  'tests/test_release_evidence.py' \
  'tests/test_release_acceptance_workflow.py' \
  'tests/test_container_packaging_contract.py' \
  'tests/test_packaging_metadata.py' \
  'tests/test_release_evidence_documentation.py'
do
  printf '\n===== %s =====\n' "$path"
  git show "${head_sha}:${path}" | nl -ba
done

echo '== Exact-head docs: release boundaries and symlink semantics =='
for path in \
  'docs/doctoring/reproducible-release-evidence.md' \
  'docs/adr/0003-reproducible-release-evidence.md'
do
  printf '\n===== %s =====\n' "$path"
  git show "${head_sha}:${path}" |
    rg -n -C 4 'stack|base|merge|retarget|head|stale|SLSA|artifact|manifest|uv|SOURCE_DATE_EPOCH|permission|pin|symlink' || true
done

echo '== Focused exact base-to-head diff =='
git diff --no-ext-diff --unified=55 "$base_sha" "$head_sha" -- \
  'pg_llm_batch/release_evidence.py' \
  '.github/workflows/release-acceptance.yml' \
  '.github/workflows/ci.yml' \
  'pyproject.toml' 'uv.lock' 'Dockerfile' \
  'tests/test_release_evidence.py' \
  'tests/test_release_acceptance_workflow.py' \
  'tests/test_container_packaging_contract.py' \
  'tests/test_packaging_metadata.py' \
  'tests/test_release_evidence_documentation.py' \
  'docs/doctoring/reproducible-release-evidence.md' \
  'docs/adr/0003-reproducible-release-evidence.md'

echo '== Exact workflow run provenance =='
for run_id in 31067011861 31067011832; do
  gh api "repos/${repo}/actions/runs/${run_id}" \
    --jq '{id,name,head_sha,event,status,conclusion,workflow_id,html_url,created_at,updated_at}'
done

echo '== Current pull request state, informational only =='
gh pr view 55 --repo "$repo" \
  --json headRefOid,baseRefOid,isDraft,mergeCommit,statusCheckRollup \
  --jq '{headRefOid,baseRefOid,isDraft,mergeCommit,statusCheckRollup:[.statusCheckRollup[]|{name,status,conclusion,headSha}]}'

Length of output: 4856


🏁 Script executed:

#!/bin/bash
set -euo pipefail

repo='ContextualWisdomLab/pg-llm-batch'
head_sha='3660bb9edd6351a9c02d9507f08ed647ddbf0d3a'
base_sha='27a2e6386e956986405ae6c7fb6ed9f615a433ae'

print_file() {
  path="$1"
  printf '\n===== %s =====\n' "$path"
  git show "${head_sha}:${path}" | awk '{printf "%5d  %s\n", NR, $0}'
}

echo '== Exact-head verifier, workflows, configuration, and tests =='
for path in \
  'pg_llm_batch/release_evidence.py' \
  '.github/workflows/release-acceptance.yml' \
  '.github/workflows/ci.yml' \
  'pyproject.toml' \
  'Dockerfile' \
  'tests/test_release_evidence.py' \
  'tests/test_release_acceptance_workflow.py' \
  'tests/test_container_packaging_contract.py' \
  'tests/test_packaging_metadata.py' \
  'tests/test_release_evidence_documentation.py'
do
  print_file "$path"
done

echo '== Exact-head documentation excerpts =='
for path in \
  'docs/doctoring/reproducible-release-evidence.md' \
  'docs/adr/0003-reproducible-release-evidence.md'
do
  printf '\n===== %s =====\n' "$path"
  git show "${head_sha}:${path}" |
    grep -n -E -C 4 'stack|base|merge|retarget|head|stale|SLSA|artifact|manifest|uv|SOURCE_DATE_EPOCH|permission|pin|symlink' || true
done

echo '== Focused exact base-to-head diff =='
git diff --no-ext-diff --unified=45 "$base_sha" "$head_sha" -- \
  'pg_llm_batch/release_evidence.py' \
  '.github/workflows/release-acceptance.yml' \
  '.github/workflows/ci.yml' \
  'pyproject.toml' 'uv.lock' 'Dockerfile' \
  'tests/test_release_evidence.py' \
  'tests/test_release_acceptance_workflow.py' \
  'tests/test_container_packaging_contract.py' \
  'tests/test_packaging_metadata.py' \
  'tests/test_release_evidence_documentation.py' \
  'docs/doctoring/reproducible-release-evidence.md' \
  'docs/adr/0003-reproducible-release-evidence.md'

echo '== Precise control references =='
git grep -nE \
  'uv==0\.12\.1|uv_build==0\.12\.1|SOURCE_DATE_EPOCH|PYTHONHASHSEED|TZ=|LC_ALL=|git archive|persist-credentials|permissions:|release-manifest|sha256|symlink|is_file|is_symlink|islice|count mismatch|exactly two' \
  "$head_sha" -- \
  '.github' 'pyproject.toml' 'Dockerfile' 'pg_llm_batch' 'tests' 'docs' || true

echo '== Exact workflow run provenance =='
for run_id in 31067011861 31067011832; do
  gh api "repos/${repo}/actions/runs/${run_id}" \
    --jq '{id,name,head_sha,event,status,conclusion,workflow_id,html_url,created_at,updated_at}'
done

echo '== Current pull request state, informational only =='
gh pr view 55 --repo "$repo" \
  --json headRefOid,baseRefOid,isDraft,mergeCommit,statusCheckRollup \
  --jq '{headRefOid,baseRefOid,isDraft,mergeCommit,statusCheckRollup:[.statusCheckRollup[]|{name,status,conclusion,headSha}]}'

Length of output: 140961


@seonghobae I found no actionable implementation finding on exact head 3660bb9edd6351a9c02d9507f08ed647ddbf0d3a against exact base 27a2e6386e956986405ae6c7fb6ed9f615a433ae.

The supplied base is an ancestor of the supplied head.

The prior count-diagnostic finding is addressed. pg_llm_batch/release_evidence.py scans at most three entries with islice. If the count is not two, it emits one fixed error message before it derives entry names. If there are exactly two entries with wrong artifact kinds, it sorts that bounded set before it reports names.

I also verified these controls:

  • The verifier rejects symlinked directories, symlinked artifacts, and non-regular artifacts.
  • The verifier validates artifact distribution and version identities.
  • SHA-256 reads artifacts in 1 MiB chunks.
  • The manifest writer rejects a symlinked destination and each symlinked parent component before mkdir().
  • The manifest writer rejects an existing temporary file or temporary symlink before os.open() and atomic replacement.
  • The workflow checks out the exact PR head with persisted credentials disabled and contents: read permissions.
  • The workflow pins uv to 0.12.1 and the PEP 517 backend to uv_build==0.12.1.
  • The workflow derives SOURCE_DATE_EPOCH from the checked-out head and sets PYTHONHASHSEED, TZ, and LC_ALL.
  • The workflow builds two separate git archive source trees.
  • Workflow actions and container images use immutable SHA pins.
  • The container build supplies LICENSE and NOTICE before dependency synchronization.
  • The operator documentation rejects predecessor-head and stale-base evidence.

Release Acceptance run 31067011861 and CI run 31067011832 completed successfully for this exact head.

The PR is still a draft. GitHub currently reports mergeCommit: null. I did not treat the successful stacked-base checks or absent default-branch workflows as final merge evidence. Fresh GitHub-generated merge-commit evidence is still required after the base changes or the PR is retargeted to integrated main.

You are interacting with an AI system.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant