Skip to content

test(reliability): bound release-index Git metadata lookup - #676

Closed
seonghobae wants to merge 0 commit into
mainfrom
fix/release-evidence-index-git-metadata-timeout-555
Closed

test(reliability): bound release-index Git metadata lookup#676
seonghobae wants to merge 0 commit into
mainfrom
fix/release-evidence-index-git-metadata-timeout-555

Conversation

@seonghobae

Copy link
Copy Markdown
Contributor

Reliability gap

Advances #555 with the previously uncovered release-evidence-index local Git metadata subprocess slice from exact protected main a7be9eb8ca0b807364173f8471fc300e3cf9350f.

scripts/build_release_evidence_index.py::_source_commit() invokes git rev-parse HEAD without a per-operation timeout and catches every exception into "unknown". A hung local Git child can therefore consume runner capacity until an outer job timeout, while subprocess.TimeoutExpired is silently converted into apparently usable provenance.

Intentional fail-first contract

Exact test-only head: 82d168bb8921c5fddb7941cf53ada5864455f481, directly based on protected main.

tests/test_release_evidence_index_git_metadata_timeout.py requires:

  • one package-owned positive local-Git deadline no greater than 30 seconds;
  • that deadline to be passed to the existing subprocess.run(["git", "rev-parse", "HEAD"], ...) call;
  • TimeoutExpired to fail closed as exact stable RuntimeError("source commit lookup timed out") without reflecting command/stdout/stderr/repo-path content;
  • successful lookup to preserve stripped commit output; and
  • ordinary non-timeout Git failures to preserve the historical "unknown" fallback.

Current protected-main code is expected to fail at the actual _source_commit() production boundary because it passes no timeout and broadly swallows timeout. Setup/import/fixture/changelog failure is not valid RED.

Minimum GREEN after exact RED proof

  1. Add a clearly named package-owned local Git metadata deadline in scripts/build_release_evidence_index.py and pass it through subprocess.run(timeout=...).
  2. Catch subprocess.TimeoutExpired explicitly before the ordinary fallback and raise RuntimeError("source commit lookup timed out") from None; never echo child-controlled details.
  3. Preserve success and the existing "unknown" fallback for ordinary non-timeout Git/OS failure.
  4. Keep artifact coverage, digest/version semantics, procurement evidence interpretation, and JSON/report construction unchanged.
  5. After product GREEN add concise reliability: add operation-specific subprocess deadlines and timeout evidence #555 reliability doctoring grounded in current Python subprocess/GitHub Actions primary docs plus an authoritative changelog fragment, render/check managed CHANGELOG.md, and require exact changed-production statement/branch coverage.
  6. Require one unchanged final head with focused/full Python, Rust/PyO3, package/reinstall/release acceptance, GPU/fuzz, Security/SAST, current-head review and repository governance.

This slice is disjoint from active #666/#667/#669/#670/#675/#578/#657 and replaces no canonical #604/#621 documentation. No psychometric/numerical formula, dependency/workflow, package version/release, valuation semantics, hosted state or reviewer authority change belongs here.

Keep Draft until RED→GREEN history and final gates are complete.

@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@seonghobae, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 11 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 695f9b12-dd3f-4ae2-950f-bde7a2b4e5aa

📥 Commits

Reviewing files that changed from the base of the PR and between c0cb88c and 091900e.

📒 Files selected for processing (1)
  • tests/test_release_evidence_index_git_metadata_timeout.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copy link
Copy Markdown
Contributor Author

@opencode-agent address

Exact bounded GREEN implementation for issue #555 / Draft #676. Immediately before any write, refetch exact source head 82d168bb8921c5fddb7941cf53ada5864455f481, independently resolve protected main (currently a7be9eb8ca0b807364173f8471fc300e3cf9350f), inspect current comments/reviews, and refetch scripts/build_release_evidence_index.py plus the fail-first test. Proceed only if source/base remain compatible and no other writer targets this branch/path; abort rather than race.

Exact fail-first evidence is valid. CI run 31432253318, Python job 93598206449, synthetic merge 25d9c4e60dc64d5ba7d1ebd028d262e6a7a2f9f5 completed 2 failed, 2949 passed, 2 skipped. The ONLY failures are the two intended tests/test_release_evidence_index_git_metadata_timeout.py contracts: _source_commit() swallows TimeoutExpired and successful lookup receives no bounded timeout=. The ordinary non-timeout fallback test is GREEN. Package/build/reinstall/release-acceptance/enterprise-readiness and GPU-smoke are GREEN; Security Scan 31432253160 and SAST 31432254093 are GREEN. The known _log_sigmoid warning is unrelated #583/#564 and must not be touched.

Implement only the root-cause fix:

  1. Add one documented package-owned local Git metadata deadline in (0, 30] seconds and pass it as timeout= to the existing subprocess.run(["git", "rev-parse", "HEAD"], ...) call.
  2. Catch subprocess.TimeoutExpired before the ordinary fallback and raise exactly RuntimeError("source commit lookup timed out") from None. Never reflect command/stdout/stderr/path or other caller-controlled content.
  3. Preserve successful stripped SHA behavior and preserve the current "unknown" fallback for ordinary non-timeout Git/OS failures.
  4. Do not alter artifact inventory, hashes, version/schema semantics, procurement evidence interpretation, JSON construction, acquisition-readiness claims, or unrelated subprocess paths.
  5. Strengthen focused tests only if needed for exact redaction/deadline behavior. After product GREEN add concise reliability: add operation-specific subprocess deadlines and timeout evidence #555 reliability doctoring grounded in current Python 3.14 subprocess timeout and GitHub Actions timeout primary documentation, add an authoritative docs/changelog.d fragment, deterministically render/check managed CHANGELOG.md, run focused tests plus git diff --check, then the full applicable suite.
  6. Require meaningful changed-production statement/branch coverage and complete docs. Keep Draft and stop source writes after one coherent verified update; final current-head CI/Security/SAST/review/Ready/merge returns to the maintainer loop.

Do not touch numerical code, dependencies/workflows, package version/release, canonical #604/#621 docs, hosted state, provider credentials or reviewer authority.

@seonghobae
seonghobae force-pushed the fix/release-evidence-index-git-metadata-timeout-555 branch from 82d168b to b46f022 Compare August 11, 2026 05:01
@seonghobae
seonghobae marked this pull request as ready for review August 11, 2026 05:01
@seonghobae
seonghobae force-pushed the fix/release-evidence-index-git-metadata-timeout-555 branch from b46f022 to 091900e Compare August 11, 2026 05:17
@seonghobae seonghobae closed this Aug 11, 2026
@seonghobae
seonghobae force-pushed the fix/release-evidence-index-git-metadata-timeout-555 branch from 091900e to 0b2ec51 Compare August 11, 2026 05:31
@github-actions

Copy link
Copy Markdown
Contributor

자동 정리: base 대비 실제 변경(diff)이 0건이라 이 PR을 닫습니다. 변경을 추가한 뒤 reopen하세요.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant