Skip to content

test(reliability): bound Figma-evidence Git metadata lookup - #677

Closed
seonghobae wants to merge 1 commit into
mainfrom
fix/figma-evidence-git-metadata-timeout-555
Closed

test(reliability): bound Figma-evidence Git metadata lookup#677
seonghobae wants to merge 1 commit into
mainfrom
fix/figma-evidence-git-metadata-timeout-555

Conversation

@seonghobae

Copy link
Copy Markdown
Contributor

Reliability gap

Advances #555 with the previously uncovered Figma-evidence-sync local Git metadata subprocess slice from exact protected main a7be9eb8ca0b807364173f8471fc300e3cf9350f.

scripts/build_figma_evidence_sync.py::_source_commit() invokes git rev-parse HEAD without a per-operation timeout and catches every exception into "unknown". A hung local Git child can therefore consume runner capacity until an outer job timeout, while subprocess.TimeoutExpired is silently converted into apparently usable provenance.

Intentional fail-first contract

Exact test-only head: c3d3772fa7fb6efe6f255806d118cb9bc71282a3, directly based on protected main.

tests/test_figma_evidence_git_metadata_timeout.py requires:

  • one package-owned positive local-Git deadline no greater than 30 seconds;
  • that deadline to be passed to the existing subprocess.run(["git", "rev-parse", "HEAD"], ...) call;
  • TimeoutExpired to fail closed as exact stable RuntimeError("source commit lookup timed out") without reflecting command/stdout/stderr/repo-path content;
  • successful lookup to preserve stripped commit output; and
  • ordinary non-timeout Git failures to preserve the historical "unknown" fallback.

Current protected-main code is expected to fail at the actual _source_commit() production boundary because it passes no timeout and broadly swallows timeout. Setup/import/fixture/changelog failure is not valid RED.

Minimum GREEN after exact RED proof

  1. Add a clearly named package-owned local Git metadata deadline in scripts/build_figma_evidence_sync.py and pass it through subprocess.run(timeout=...).
  2. Catch subprocess.TimeoutExpired explicitly before the ordinary fallback and raise RuntimeError("source commit lookup timed out") from None; never echo child-controlled details.
  3. Preserve success and the existing "unknown" fallback for ordinary non-timeout Git/OS failure.
  4. Keep Figma packet/frame/token coverage semantics and all buyer-evidence interpretation unchanged; this is process reliability only.
  5. After product GREEN add concise reliability: add operation-specific subprocess deadlines and timeout evidence #555 reliability doctoring grounded in current Python subprocess/GitHub Actions primary docs plus an authoritative changelog fragment, render/check managed CHANGELOG.md, and require exact changed-production statement/branch coverage.
  6. Require one unchanged final head with focused/full Python, Rust/PyO3, package/reinstall/release acceptance, GPU/fuzz, Security/SAST, current-head review and repository governance.

This slice is disjoint from active #666/#667/#669/#670/#675/#676/#578/#657 and replaces no canonical #604/#621 documentation. No psychometric/numerical formula, dependency/workflow, package version/release, valuation semantics, hosted state or reviewer authority change belongs here.

Keep Draft until RED→GREEN history and final gates are complete.

@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@seonghobae, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 1 minute

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: b73be90f-9042-4e24-befc-3a3047d573a0

📥 Commits

Reviewing files that changed from the base of the PR and between 384516b and ba27d55.

📒 Files selected for processing (2)
  • scripts/build_figma_evidence_sync.py
  • tests/test_figma_evidence_git_metadata_timeout.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copy link
Copy Markdown
Contributor Author

@opencode-agent address

Exact bounded GREEN implementation for issue #555 / Draft #677. Immediately before any write, refetch exact source head c3d3772fa7fb6efe6f255806d118cb9bc71282a3, independently resolve protected main (currently a7be9eb8ca0b807364173f8471fc300e3cf9350f), inspect current comments/reviews, and refetch scripts/build_figma_evidence_sync.py plus the fail-first test. Proceed only if source/base remain compatible and no other writer targets this branch/path; abort rather than race.

Exact fail-first evidence is valid. CI run 31432318167, Python job 93598413035, synthetic merge 2a54eca7630b4ab5e98992613f28245630727454 completed 2 failed, 2949 passed, 2 skipped. The ONLY failures are the two intended tests/test_figma_evidence_git_metadata_timeout.py contracts: _source_commit() swallows TimeoutExpired and successful lookup receives no bounded timeout=. The ordinary non-timeout fallback test is GREEN. Rust/PyO3, package/build/reinstall/release acceptance/enterprise-readiness, GPU-smoke and fuzz are GREEN; Security Scan 31432318233 and SAST 31432317916 are GREEN. The known _log_sigmoid warning is unrelated #583/#564 and must not be touched.

Implement only the root-cause fix:

  1. Add one documented package-owned local Git metadata deadline in (0, 30] seconds and pass it as timeout= to the existing subprocess.run(["git", "rev-parse", "HEAD"], ...) call.
  2. Catch subprocess.TimeoutExpired before the ordinary fallback and raise exactly RuntimeError("source commit lookup timed out") from None. Never reflect command/stdout/stderr/path or other caller-controlled content.
  3. Preserve successful stripped SHA behavior and preserve the current "unknown" fallback for ordinary non-timeout Git/OS failures.
  4. Do not alter Figma packet/frame/token coverage, component/variable evidence interpretation, JSON/HTML output semantics, or unrelated subprocess behavior.
  5. Strengthen focused tests only if needed for exact redaction/deadline behavior. After product GREEN add concise reliability: add operation-specific subprocess deadlines and timeout evidence #555 reliability doctoring grounded in current Python 3.14 subprocess timeout and GitHub Actions timeout primary documentation, add an authoritative docs/changelog.d fragment, deterministically render/check managed CHANGELOG.md, run focused tests plus git diff --check, then the full applicable suite.
  6. Require meaningful changed-production statement/branch coverage and complete docs. Keep Draft and stop source writes after one coherent verified update; final current-head CI/Security/SAST/review/Ready/merge returns to the maintainer loop.

Do not touch numerical code, dependencies/workflows, package version/release, canonical #604/#621 docs, hosted state, provider credentials or reviewer authority.

Copy link
Copy Markdown
Contributor Author

@opencode-agent address

Fresh single-writer rebind for Draft #677 after protected main advanced. The earlier handoff 5246126872 was bound to main a7be9eb8ca0b807364173f8471fc300e3cf9350f, has no receipt/reaction, and is stale. Immediately before writing, refetch and proceed only if source head remains exactly c3d3772fa7fb6efe6f255806d118cb9bc71282a3, protected main is exactly 36407d6bee04da3bf2310b6377c9fe1c80e4b982, scripts/build_figma_evidence_sync.py still has the unbounded _source_commit() call, the fail-first tests still express the same boundary, and no other writer is active. Abort rather than race if any identity changes.

Revalidate that current protected main has not independently repaired this script. If the defect persists, implement only the root-cause correction: add one documented package-owned local-Git metadata deadline in (0, 30]; pass it to the existing subprocess.run(["git", "rev-parse", "HEAD"], ..., timeout=...); catch subprocess.TimeoutExpired explicitly and raise exact non-reflective RuntimeError("source commit lookup timed out") from None; preserve successful stripped output and the ordinary non-timeout "unknown" fallback. Do not change Figma packet/frame/token semantics, valuation fields, dependencies/workflows, version/release, canonical #604/#621 docs, or unrelated subprocess classes.

Run focused timeout tests first and then applicable full validation on one unchanged head. After product GREEN, add concise #555 reliability doctoring plus an authoritative changelog fragment, deterministically render/check CHANGELOG.md, and recreate CI/Security/SAST/current-head review evidence against the live base. Keep Draft until all gates are terminal.

@seonghobae
seonghobae force-pushed the fix/figma-evidence-git-metadata-timeout-555 branch from c3d3772 to fbfa17a Compare August 11, 2026 05:01
@seonghobae
seonghobae marked this pull request as ready for review August 11, 2026 05:01
@seonghobae
seonghobae force-pushed the fix/figma-evidence-git-metadata-timeout-555 branch 2 times, most recently from 0d3ba94 to 206c41d Compare August 11, 2026 05:30
@seonghobae seonghobae closed this Aug 11, 2026
@seonghobae
seonghobae force-pushed the fix/figma-evidence-git-metadata-timeout-555 branch from 206c41d to 0b2ec51 Compare August 11, 2026 05:43
@github-actions

Copy link
Copy Markdown
Contributor

자동 정리: base 대비 실제 변경(diff)이 0건이라 이 PR을 닫습니다. 변경을 추가한 뒤 reopen하세요.

@seonghobae seonghobae reopened this Aug 11, 2026
@seonghobae
seonghobae force-pushed the fix/figma-evidence-git-metadata-timeout-555 branch from 206c41d to d4fce12 Compare August 11, 2026 05:56
@seonghobae
seonghobae force-pushed the fix/figma-evidence-git-metadata-timeout-555 branch from d4fce12 to ba27d55 Compare August 11, 2026 06:28
@seonghobae

Copy link
Copy Markdown
Contributor Author

Superseded by unified Git metadata timeout fix PR (see fix/git-metadata-timeouts-unified).

@seonghobae seonghobae closed this Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant