Skip to content

fix(ci): exempt draft PRs from the opencode-review current-head verdict gate - #1443

Closed
seonghobae wants to merge 16 commits into
mainfrom
claude/opencode-review-draft-gate-fix
Closed

fix(ci): exempt draft PRs from the opencode-review current-head verdict gate#1443
seonghobae wants to merge 16 commits into
mainfrom
claude/opencode-review-draft-gate-fix

Conversation

@seonghobae

Copy link
Copy Markdown
Contributor

The bug

.github/workflows/opencode-review.yml's required check job (opencode-review-target, displayed as the opencode-review check) triggers on pull_request_target: types: [opened, synchronize, reopened, ready_for_review, closed]. Its one step, "Fail closed without a current-head OpenCode verdict", unconditionally demands an APPROVED/CHANGES_REQUESTED review from opencode-agent on the current head, with no draft handling:

if [ "${{ github.event.action }}" = "closed" ]; then
  echo "PR closed; a current-head OpenCode verdict is not required."
  exit 0
fi
if [ -z "${PR_NUMBER:-}" ] || [ -z "${HEAD_SHA:-}" ]; then
  echo "::error::Missing PR number or head SHA; cannot verify a current-head OpenCode verdict."
  exit 1
fi
reviews="$(gh api --paginate "repos/${TARGET_REPOSITORY}/pulls/${PR_NUMBER}/reviews")"
verdict="$(... jq query for an APPROVED/CHANGES_REQUESTED review from opencode-agent on the exact HEAD_SHA ...)"
if [ -z "$verdict" ]; then
  echo "::error::No APPROVED or CHANGES_REQUESTED from opencode-agent on the current head. ..."
  exit 1
fi

Meanwhile scripts/ci/pr_review_merge_scheduler.py — the thing that actually issues the repository_dispatch event that would eventually cause opencode-agent to post a verdict — deliberately never requests a review for a draft PR (line 2401-2402):

if pr.get("isDraft"):
    return Decision(number, "skip", "draft PR")

Net effect: every draft PR in the org showed this required check as a hard exit 1 failure on every single push, forever, until marked ready for review — not a transient/pending state, an active failure with a scary-looking error message every time. This was independently observed recurring identically on two different draft PRs across two different repos (ContextualWisdomLab/.github#1437 and ContextualWisdomLab/contextual-orchestrator#922).

The fix

Add a draft-aware early-exit to the same step, mirroring the existing closed early-exit exactly in style and placement (right after it, before the PR_NUMBER/HEAD_SHA check):

if [ "${{ github.event.pull_request.draft }}" = "true" ]; then
  echo "PR is a draft; a current-head OpenCode verdict is not required until it is marked ready for review."
  exit 0
fi

This preserves the org's own stated design principle (9550c18: "a required-workflow check must never depend on event payload fields to materialize") — that principle governs the required-workflow-bootstrap job's materialization (confirmed via scripts/ci/test_strix_quick_gate.sh's job-scoped if: scan), which is unaffected here. The opencode-review-target job itself still always runs unconditionally and always reports a status; only its internal bash logic decides pass/fail, exactly like the pre-existing closed branch.

Why this is safe / doesn't weaken the real gate:

  • Once a PR is marked ready for review, ready_for_review and subsequent synchronize events carry draft: false, so the check goes back to genuinely requiring a current-head verdict, exactly as today.
  • A draft PR still cannot be merged via GitHub's own draft mechanism regardless of this check's status, so this closes a false-alarm gap without opening a real merge-bypass gap.
  • Searched for duplicates of this pattern (same jq query text / same "No APPROVED or CHANGES_REQUESTED from opencode-agent" string) across every workflow in the repo: this is the only occurrence. opencode-review-dispatch.yml has an unrelated job also named opencode-review-target (the privileged reviewer itself, repository_dispatch-triggered) with no equivalent unconditional-demand pattern. noema-review.yml is a different shape entirely — it performs the review itself rather than demanding a pre-existing verdict — so it isn't affected by this bug.

Tests

Added shell-level regression coverage in tests/test_opencode_required_verdict_regression.py that extracts the production step's literal bash body from the YAML (mirroring the existing _extract_run_block pattern already used in tests/test_opencode_workflow_shell_syntax.py), substitutes the two inline ${{ github.* }} expressions the way GitHub Actions would, and executes it directly against fake gh binaries:

  • draft PR short-circuits before any Reviews API call is ever attempted (a "refuse to be invoked" fake gh proves this), across opened/synchronize/reopened
  • closed still takes precedence over draft (ordering regression guard)
  • a non-draft, ready_for_review PR with a matching current-head APPROVED review still passes through the real gate unchanged
  • a non-draft PR with no matching review still fails closed exactly as before

Full local check suite:

coverage run -m pytest tests -q   # 1889 passed, 1 skipped (pre-existing, unrelated: missing LLVM 19 toolchain), 21 subtests passed
interrogate                        # 100.0% (pass)

Note: coverage report --show-missing shows 99% (one line in scripts/ci/pingora_edge_policy.py's changed-file pagination fallback, untouched by this PR) — confirmed byte-for-byte identical and pre-existing on a clean main checkout before any of this PR's changes, via git stash/re-run. Not introduced or worsened by this PR; flagging for visibility rather than silently masking it.

Scope

This is a narrow, self-contained fix to one required-check step. No changes to docs/pr-review-and-merge-procedure.md or PR_GOVERNANCE_AUDIT.md — neither currently describes this check's draft behavior, so neither was rendered inaccurate by this change.

Opening as draft per repo governance — OpenCode-approval + scheduler review/merge applies here same as any other PR.

Co-Authored-By: Claude Sonnet 5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01BV96rXhqoR3tYZ9AeAVur4


Generated by Claude Code

…ct gate

opencode-review.yml's required opencode-review-target check ("Fail closed
without a current-head OpenCode verdict") unconditionally demanded an
APPROVED/CHANGES_REQUESTED review from opencode-agent on the current head for
every opened/synchronize/reopened event, with no draft handling. Meanwhile
scripts/ci/pr_review_merge_scheduler.py deliberately never dispatches an
OpenCode review request for a draft PR:

    if pr.get("isDraft"):
        return Decision(number, "skip", "draft PR")

Net effect: every draft PR showed this required check as a hard exit-1
failure on every push, forever, until marked ready for review -- a permanent
false alarm, not a transient/pending state.

Add a github.event.pull_request.draft early-exit mirroring the existing
closed early-exit exactly in style and placement (right after it, before the
PR_NUMBER/HEAD_SHA check). The job still always runs and always reports a
status -- it just reports success instead of a misleading failure for a
state where a verdict was never going to be requested. Once the PR is marked
ready for review, ready_for_review and subsequent synchronize events carry
draft: false, so the real gate applies unchanged.

Adds shell-level regression coverage in
tests/test_opencode_required_verdict_regression.py that executes the
production step body directly (draft short-circuits before any Reviews API
call, closed still takes precedence over draft, and non-draft PRs still
genuinely require a verdict).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BV96rXhqoR3tYZ9AeAVur4
@coderabbitai

coderabbitai Bot commented Aug 30, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 58 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: dd205165-2afa-4a4b-89b6-e822379135d6

📥 Commits

Reviewing files that changed from the base of the PR and between 44a3c74 and c3d18f9.

📒 Files selected for processing (6)
  • .github/workflows/opencode-review.yml
  • CHANGELOG.md
  • docs/product-technical-gap-baseline.md
  • scripts/ci/test_strix_quick_gate.sh
  • tests/test_opencode_required_verdict_regression.py
  • tests/test_required_workflow_queue_contract.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae
seonghobae marked this pull request as ready for review August 30, 2026 12:04
devin-ai-integration[bot]

This comment was marked as resolved.

claude and others added 2 commits August 30, 2026 12:22
A ready PR converted back to draft with no new commit never fired the
required-workflow gate again (converted_to_draft wasn't in its trigger
list), so a previously failed opencode-review check stayed failed
forever even though the existing draft exemption would have passed it.

Add converted_to_draft to opencode-review.yml's pull_request_target
types, and update the matching contract/regression tests.

Found by Devin's automated review on PR #1443.

Copy link
Copy Markdown
Contributor Author

Status on the failing opencode-review check (run 33310503217): it failed because no opencode-agent review existed yet for this head SHA — this is a pipeline-timing gap, not a bug in this PR's diff. This PR was just converted from draft to ready-for-review; on a cross-repo target ready_for_review triggers the required-workflow gate check immediately, but the actual OpenCode review dispatch is scheduled by pr_review_merge_scheduler.py (immediately for same-repo .github PRs like this one, otherwise via its 15-minute org-wide sweep), so there's a normal window where the gate has nothing to check yet.

Separately, pushed 644255a to fix the real bug Devin found on this PR (draft-reconversion trigger gap) — see the resolved review thread. That push will itself trigger a fresh required-workflow run and review dispatch for the new head.


Generated by Claude Code

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

This report is out of date. Scroll down for Devin Review's latest report on this PR.

Devin Review found 0 new potential issues.

Devin Review

Copy link
Copy Markdown
Contributor Author

Fresh operational contradiction: explicit draft review-only request is discarded

Protected main@1d8e872487838e16a003e96e76df9300c388e258 still violates the draft review-only contract at the scheduler boundary.

On Draft PR #1450 at unchanged exact head 7779b20fd0f525c87baa6ccc156ee79c607fa9a1, a targeted @opencode-agent review request explicitly required review-only execution and prohibited branch/merge-state mutation.

Fresh exact evidence:

Current #1443 source makes the required status green for a Draft, but leaves pr_review_merge_scheduler.py's unconditional draft skip intact. That converts an explicit review-only request into no review at all. It is not acceptance for the required contract that Draft review dispatch may run while lifecycle/branch mutations remain disabled.

Please extend the TDD boundary so:

  1. an explicit targeted/mention review request for an open Draft dispatches the exact-current-head OpenCode review;
  2. the same Draft path is structurally review-only and cannot update refs, merge, enable auto-merge, or otherwise change lifecycle state;
  3. ordinary merge-queue sweeps may still skip Drafts;
  4. non-Draft verdict gating remains fail-closed; and
  5. a live post-integration Draft canary produces a formal exact-head review without leaving Draft.

The addressed converted_to_draft thread is separately resolved; this is a distinct current operational defect.

Copy link
Copy Markdown
Contributor Author

Confirmed — this is a distinct defect from the one this PR fixes. Filed and fixed in #1456: inspect_pr() returned skip: draft PR unconditionally, before ever reaching the dispatch logic, so agent-mention-opencode-dispatch.yml's already-structurally-review-only forward (it already hardcodes enable_auto_merge=false, update_branches=false, merge_mode=disabled) was silently discarded for drafts.

New opt-in --allow-draft-review-dispatch (requires --pr-number, so it can never apply to the multi-PR queue sweep) routes a draft PR through a new dispatch_draft_review_only() helper that runs the same Strix-then-OpenCode gate the ready-PR pipeline uses, then returns before any merge/branch-update/auto-merge logic — gated in the workflow by a new ALLOW_DRAFT_REVIEW_DISPATCH env var keyed on client_payload.agent_invocation_key, a field only the mention-dispatch workflow ever sets. Full test suite, 100% coverage/docstrings on the changed script, and a dedicated regression test proving the review-only path never calls any merge/branch/auto-merge function. See #1456 for the itemized response to all 5 requirements.


Generated by Claude Code

…w-draft-gate-fix

# Conflicts:
#	CHANGELOG.md

Copy link
Copy Markdown
Contributor Author

Resolved a merge conflict against protected main (dozens of commits had landed since this PR's last update): pushed merge commit 43d7e893.

Only CHANGELOG.md conflicted -- both sides had independently added a new bullet under ## [Unreleased], and git's diff also happened to concatenate the tail of one bullet with the start of another during the 3-way merge, producing a confusing but mechanical split. Resolved by keeping this PR's own bullet (the draft-exemption fix), followed by every intervening bullet already on main, followed by the one shared bullet both sides converged back onto (ORCHESTRATOR_CATALOG_FAMILY_CAP default 4→8) -- verified no duplication or truncation at either boundary.

Every actual code/test/workflow file (.github/workflows/opencode-review.yml, scripts/ci/test_strix_quick_gate.sh, both tests/test_opencode_required_verdict_regression.py and tests/test_required_workflow_queue_contract.py) auto-merged cleanly with zero conflicts -- this PR's own draft-exemption logic and regression tests are unchanged by the merge.

Local full test suite is running to confirm (this repo's suite includes several deliberate 30s+ timeout-testing sleeps in test_strix_quick_gate.sh, so it runs long); will report if it surfaces anything. This PR was already fully green in its own prior session (1889 passed, 1 skipped, 100% interrogate) before this catch-up merge.


Generated by Claude Code

devin-ai-integration[bot]

This comment was marked as resolved.

opencode-review-target's earlier "Request current-head OpenCode review
execution" step still ran unconditionally (only excluding closed
events), performing its own OIDC token exchange, app-token exchange,
and repository_dispatch call under set -euo pipefail. Any transient
failure there could fail the job before the later step's draft
exemption ever ran, keeping the required check red on draft PRs during
an infrastructure outage unrelated to draft status.

Gate this step on !github.event.pull_request.draft as well, and add a
regression test asserting the exact if: condition on the raw workflow
YAML.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BV96rXhqoR3tYZ9AeAVur4

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

This report is out of date. Scroll down for Devin Review's latest report on this PR.

Devin Review found 0 new potential issues.

Devin Review

Copy link
Copy Markdown
Contributor Author

Fresh consumer canary confirming this PR's causal boundary: ContextualWisdomLab/bandscope#845 is currently Draft at exact head d1a8412bc85a875cc83fddb16608fbd73e44151e over protected develop@749511c3ad4000090048718f685c6bee6b3d2c25. Its required opencode-review job 99647255200 (run 33427139772) executed Request current-head OpenCode review execution successfully, then polled for 180 × 30 s and failed at 2026-09-01T00:05:51Z with No APPROVED or CHANGES_REQUESTED from opencode-agent on the current head. No current-head OpenCode verdict exists; predecessor OpenCode reviews are on 998b404... / d729715... only. Same-head coverage-evidence is terminal success, so this is not the old coverage blocker.

This is exactly the draft-state contradiction #1443 owns: the required workflow dispatches/waits for a verdict while the scheduler does not review Draft PRs. No BandScope-local workflow/source workaround is appropriate. Acceptance canary after #1443 reaches protected main: re-run an unchanged Draft BandScope head and require the required OpenCode workflow to materialize but skip both review dispatch and verdict polling, returning terminal success solely because pull_request.draft == true; after ready_for_review, the same workflow must again require an authenticated exact-head verdict.

Copy link
Copy Markdown
Contributor Author

Thanks — good independent confirmation this is exactly the bug this PR fixes, on a third repo now (bandscope#845, alongside the #1437/contextual-orchestrator#922 cases already cited in this PR's description).

Status: this PR itself is fully validated (full suite, both the original draft-gating fix and the follow-up "Request current-head OpenCode review execution" dispatch-step fix) and waiting on a formal current-head OpenCode verdict — Devin's latest pass shows 0 new issues, no other check is red. Once that verdict lands and this merges to main, every consumer repo under the central required-workflow ruleset (including bandscope) picks up the fix automatically on its next opencode-review run — no per-repo propagation needed. bandscope isn't in this session's repo scope, so I can't directly trigger its re-run myself; the acceptance canary you specified (an unchanged Draft head materializing the required workflow, skipping dispatch and verdict polling, terminal success purely from pull_request.draft == true) is exactly what this PR's own regression tests already assert against the production step body — I'll re-verify against a live bandscope run if/when this session or another has access to that repo.


Generated by Claude Code

@seonghobae

Copy link
Copy Markdown
Contributor Author

Contextual-Orchestrator와 관계한 것들을 같이 손보든 어쩌든 해결하세요. Bypass merge 필요하면 가능 (chicken and eggs 상황이라면) + NVIDIA NIM 만 쓰는 건 허용하지 않아요. Contextual-Orchestrator를 쓰세요. Timeout은 적어도 3시간으로 잡으세요. 120초 같은 건 당황스럽군요. Opencode와 Noema 는 Coderabbitai 및 Devin 수준으로 실제로 리뷰를 하게 하시오. Strix도 보안 리뷰를 꼼꼼하게 하도록 하시오. 특히 보안 리뷰는 전체 코드로 수행하는 것입니다. Contextual-Orchestrator는 실시간으로 빠르면서 능력이 좋은 모델에 요청을 보내어 시간을 당기시오.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Contextual-Orchestrator와 관계한 것들을 같이 손보든 어쩌든 해결하세요. Bypass merge 필요하면 가능 (chicken and eggs 상황이라면) + NVIDIA NIM 만 쓰는 건 허용하지 않아요. Contextual-Orchestrator를 쓰세요. Timeout은 적어도 3시간으로 잡으세요. 120초 같은 건 당황스럽군요. Strix가 6시간 이상 동작해서 취약점 잡는 것도 본 일이 있습니다. Opencode와 Noema 는 Coderabbitai 및 Devin 수준으로 실제로 리뷰를 하게 하시오. Strix도 보안 리뷰를 꼼꼼하게 하도록 하시오. 특히 보안 리뷰는 전체 코드로 수행하는 것입니다. Contextual-Orchestrator는 실시간으로 빠르면서 능력이 좋은 모델에 요청을 보내어 시간을 당기시오. @opencode-agent 라고 부르면 호출되는 기능도 인터넷 가이드에는 /oc 라고 나와있기 때문에 이 점도 확인해 보는 게 좋겠습니다.

Resolves a real conflict in tests/test_opencode_required_verdict_regression.py
(both branches added independent, non-overlapping tests to the same file:
this branch's draft-gate regression tests, main's #1532 poll-budget-ceiling
guard) by keeping both.

Also fixes two issues the merge exposed:
- The merged-in poll loop bound increase (180->660 attempts via #1532) turned
  test_non_draft_pr_without_a_verdict_still_fails_closed's real `sleep 30`
  calls into a ~5.5-hour test; stub `sleep` as a no-op on PATH alongside the
  existing fake `gh`.
- main is currently red on the review-dispatch blob pin and the
  security-boundary test (stale since #1533's already-merged head_sha
  warn-and-proceed change); re-pin both to match, same fix already applied
  in #1482, pending #1536.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BV96rXhqoR3tYZ9AeAVur4

Copy link
Copy Markdown
Contributor Author

Resolved the merge conflict against main (756987b5). main had moved ahead with two independent changes:

  • #1532 raised the opencode-review verdict-poll loop from 180 to 660 attempts, adding its own new regression test (test_verdict_poll_budget_covers_the_dispatched_review_jobs_own_ceiling) to the same test file this PR's draft-gate tests live in — kept both, no logic conflict.
  • A stale review-dispatch blob pin + security-boundary test (same root cause already flagged and fixed on #1482, pending #1536's merge) — re-pinned identically here.

The merge also exposed a real bug: test_non_draft_pr_without_a_verdict_still_fails_closed executes the production step's actual bash polling loop with real sleep 30 calls when no verdict is ever found. At 180 attempts that was already a slow ~90-minute test; at 660 (post-#1532) it would be ~5.5 hours. Fixed by stubbing sleep as a no-op on PATH alongside the existing fake gh — the loop's real attempt-count logic is still exercised, it just doesn't block on wall-clock time.

Validated: full suite 2137 passed, 1 skipped, 21 subtests passed, coverage report 100%, interrogate 100%, ruff check clean, git diff --check clean — no concurrent git operations this run.


Generated by Claude Code

devin-ai-integration[bot]

This comment was marked as resolved.

@seonghobae

Copy link
Copy Markdown
Contributor Author

뭔가 한 거 같긴 한데 OpenCode Review 안 되는군.

…e draft-gate fix

main's opencode-review.yml has been substantially redesigned since this
branch last synced (#1507/#1532): the old 325-minute synchronous poll
loop is gone, replaced by a fast check-once-dispatch-and-fail-closed
"Resolve current-head formal OpenCode verdict" step plus a separate
formal-receipt "wake" callback that reruns the failed job once a verdict
actually lands, instead of blocking a runner for hours. #1533's
head_sha warn-and-proceed design (which an earlier revision of this
fix's blob-pin port matched) was also reverted upstream (#1540, real
bugs found by Codex/Devin) -- restored to the original hard-fail
assertions and current blob pin.

The draft-gate exemption itself is unaffected by any of that and is
re-applied cleanly against the new three-step structure:
- "Resolve current-head formal OpenCode verdict" now exits early with
  verdict=DRAFT for a draft PR, mirroring its existing closed exit.
- "Request current-head OpenCode review execution"'s own if: also skips
  drafts, so a transient OIDC/dispatch failure can't turn a draft PR's
  check red before the exemption runs.
- The now-trivial "Fail closed without a current-head OpenCode verdict"
  step (no gh calls or loop left in it at all) treats VERDICT=DRAFT the
  same as VERDICT=CLOSED.
- converted_to_draft added to the trigger types, so a ready PR converted
  back to draft with no new commit still gets a fresh run.

tests/test_opencode_required_verdict_regression.py's old _run_step
helper and its six tests assumed the removed monolithic polling step;
replaced with _run_verdict_step/_run_fail_closed_step matching the new
split, keeping the same draft/closed/ready-for-review coverage.

Full suite: 2216 passed, 1 skipped, 21 subtests. Ruff, interrogate,
YAML, and shell-syntax checks clean.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BV96rXhqoR3tYZ9AeAVur4

Copy link
Copy Markdown
Contributor Author

업데이트: 제 이전 재실행 판단이 틀렸습니다 — 단순 큐 congestion이 아니라, mainopencode-review.yml이 그 사이 완전히 재설계되어 있었습니다 (#1507이 325분 동기 폴링 루프를 제거하고, "한 번 확인 → dispatch → 즉시 실패, 이후 formal receipt가 실패한 job을 wake" 방식으로 바뀜; 별개로 #1533도 실제 버그가 발견되어 #1540으로 revert됨). 이 브랜치는 그 설계 변경 이전 상태로 멈춰 있었던 겁니다.

a279b458에서 새 3-step 구조(Resolve verdict → Request dispatch → Fail closed)에 맞춰 draft-gate 예외 처리를 처음부터 다시 구현하고, 낡은 폴링 기반 테스트 6개를 새 구조에 맞는 걸로 교체했습니다. 전체 테스트 스위트 2216 passed, ruff/interrogate/YAML/bash 문법 검사 모두 clean. mergeable_state도 이제 "blocked"(conflict 없음, main과 완전히 동기화됨)로 정상입니다.

새 설계는 빠르게 실패/성공하도록 되어 있어서(더 이상 몇 시간씩 폴링하지 않음), 큐 슬롯만 받으면 몇 분 안에 결과가 나올 것으로 예상합니다. 정상적인 required check 통과를 먼저 지켜보고, 그래도 계속 막히면 말씀하신 chicken-and-egg bypass를 쓰겠습니다.


Generated by Claude Code

devin-ai-integration[bot]

This comment was marked as resolved.

…closed state

Devin review on #1443: a manual re-run of an old workflow run (e.g. a stale
converted_to_draft run) replays that event's stored github.event.* fields
verbatim, so a since-ready, unreviewed PR at the same head SHA could pass
the required opencode-review check on a stale "still draft" reading. The
verdict step now decides closed/draft from the pull request's live state
via gh api instead of the triggering event's payload, and fails closed if
that lookup itself fails.
devin-ai-integration[bot]

This comment was marked as resolved.

Devin review on #1443: the "Request current-head OpenCode review
execution" step's if: still combined github.event.action/
github.event.pull_request.draft with steps.verdict.outputs.verdict == ''.
Once the verdict step resolves closed/draft from live PR state, those
stale-payload conjuncts became a liability: a manual re-run of an old
closed/draft-era job could still suppress the dispatch for a since-
reopened/ready PR at the same head SHA, leaving the required check red
with no review ever requested. Gate on the live verdict signal alone.
devin-ai-integration[bot]

This comment was marked as resolved.

Devin review on #1443: even with the dispatch step correctly enabled from
the live verdict, it still built its repository_dispatch payload's
pr_base_ref/pr_base_sha/pr_head_ref/pr_head_sha from
github.event.pull_request.* -- the same stale-payload source the prior two
fixes removed from the pass/fail decision. On a manual re-run of an old job
whose base branch has since advanced, opencode-review-dispatch.yml's live
validate-pr-metadata check hard-rejects that stale base_sha, so the
dispatch would fail even though the verdict step correctly decided one was
needed. The verdict step now exposes base_ref/base_sha/head_ref/head_sha as
step outputs from the same live gh api response it already uses for
closed/draft, and the dispatch step builds its payload from those instead.

Copy link
Copy Markdown
Contributor Author

Current-main convergence: protected main is 2436454e3a969a282b5edc7303a485ccd37c3e9f; this branch is 12 ahead / 1 behind with merge base 9b57e4bb.... All currently visible review threads are now resolved, and the current source has the important live-state hardening: closed/draft decision and dispatch base/head metadata are fetched from the live PR API, so stale reruns cannot grant a draft exemption or dispatch stale base SHA. Please merge current protected main into this branch non-destructively (no rebase/force), preserve the effective draft-wakeup fix, run the exact regression/full suite, and regenerate exact-head review/check evidence. This remains high leverage for #1531 because drafts currently allocate required OpenCode review work they cannot merge from.

Copy link
Copy Markdown
Contributor Author

Traceability while converging this queue-reduction lane: add one concise append-only dated entry to docs/product-technical-gap-baseline.md after current-main synchronization. Record the buyer/control-plane effect (draft PRs no longer auto-dispatch required OpenCode review work, converted_to_draft refreshes the check, live PR state/base/head are re-read so stale workflow reruns cannot grant exemptions or dispatch stale metadata), and relate it to #1531 queue pressure. Do not rewrite historical snapshots or add unrelated product gaps.

Copy link
Copy Markdown
Contributor Author

Done. Merged current protected main (44a3c740, which also picks up 2436454e and #1541) into this branch non-destructively (regular merge commit, no rebase/force) at 7dbd94c2 — clean auto-merge, no conflicts. All three live-state hardening fixes (closed/draft decision, dispatch if:, and dispatch base/head metadata — all sourced from a live gh api PR fetch rather than the triggering event's stored payload) are preserved. Full suite revalidated on the merged head: 2238 passed, 1 skipped, 21 subtests; ruff clean; interrogate 100%; git diff --check clean.


Generated by Claude Code

Append-only dated entry documenting the three-round live-state hardening
on .github#1443 (draft-gate exemption, then two Devin-flagged stale-event-
payload gaps in the dispatch if: and dispatch payload), tied to #1531's
required-workflow queue-pressure tracking.

Copy link
Copy Markdown
Contributor Author

Added the traceability entry to docs/product-technical-gap-baseline.md at d3a31ff8 — append-only, dated 2026-09-01, right before §5 (matching the existing log-entry placement convention). Covers the buyer/control-plane effect (wasted required-workflow queue allocation on draft PRs, tied to #1531) and the three review-driven rounds (draft-gate exemption → live-state decision → live-state dispatch payload). No historical entries rewritten; no unrelated gaps added. test_product_technical_gap_baseline.py (5 passed) and the full suite (2238 passed, 1 skipped, 21 subtests) both green on this head.


Generated by Claude Code

devin-ai-integration[bot]

This comment was marked as resolved.

Copy link
Copy Markdown
Contributor Author

@opencode-agent Please close the current exact-head live-state gap rather than merely weakening the new baseline prose. Devin is correct: formal-review matching still uses event-derived HEAD_SHA, while exemptions and dispatch metadata use the live PR fetch. Use the same validated live head.sha from the verdict step for review matching, so a stale rerun can neither accept an approval for a predecessor event head nor miss an approval on the actual live head. Keep the live base/head outputs already used by dispatch, update the regression to give event HEAD_SHA and live head SHA different values, and assert matching is against live head only. Then correct the baseline entry to state exactly that exemptions, verdict matching, and dispatch identity are all live-API-derived. Re-run focused/full tests and request a fresh exact-head review on the resulting commit; no force-push or predecessor evidence transfer.

Copy link
Copy Markdown
Contributor Author

@opencode-agent review

Fresh substantive review on exact head 413a362c7ace1d4a662bdf8966172b56c1e432b9. All current Devin threads are resolved. Verify the intended split explicitly: live PR API state/base/head controls draft/closed exemption and any new dispatch metadata; the formal review lookup remains bound to this workflow run's event head because the resulting required check is attached to that commit. Confirm no stale rerun can satisfy a different head, no automatic dispatch occurs for a live draft, and explicit exact-head draft mentions remain supported. Predecessor reviews/checks do not transfer.

Owner direction, agreeing with and extending a Devin review comment on
#1443: the formal-review-matching jq query still keyed off the triggering
event's own HEAD_SHA, the one event-derived value the prior three rounds
hadn't touched. A stale rerun's event payload could therefore match an
approval that was only ever valid for a predecessor head, or miss a real
approval already posted against the actual live head. The verdict step now
matches reviews against the live head.sha from the same gh api fetch it
already uses for closed/draft/dispatch metadata. Adds regression coverage
for both directions, corrects the CHANGELOG/gap-baseline prose accordingly.

Copy link
Copy Markdown
Contributor Author

Done at c3d18f95. Implemented exactly as directed: the formal-review-matching jq query now matches against the live head.sha captured in the same gh api repos/.../pulls/<number> fetch the verdict step already uses for closed/draft and dispatch metadata (pr_head_sha shell variable, reused for --arg sha), not the event-derived HEAD_SHA. Added test_verdict_step_matches_reviews_against_the_live_head_not_the_event_head, which gives the event head_sha and the live head.sha different values and asserts both directions: an approval posted only for the stale event head does not satisfy the check, and an approval posted for the actual live head does — even though the event payload names a different head in both cases. Corrected CHANGELOG.md and docs/product-technical-gap-baseline.md to state that the exemption decision, the review match, and the dispatch payload are all now live-API-derived (no remaining deliberately-event-derived value). Full suite: 2239 passed, 1 skipped, 21 subtests; ruff clean; interrogate 100%; git diff --check clean. No force-push; regular commit on top of the prior history.

@opencode-agent please review this exact head (c3d18f95).


Generated by Claude Code

Copy link
Copy Markdown
Contributor Author

Flagging a timing/ordering note rather than acting further: this review request (04:19:09) references exact head 413a362c and describes the formal-review lookup as intentionally staying bound to the event head. That's the design I originally shipped and defended — but a separate comment at 04:14:30 gave the opposite, more specific direction ("neither accept an approval for a predecessor event head nor miss an approval on the actual live head"), which I implemented and pushed at c3d18f95: the verdict step now matches formal reviews against the live head.sha from the same gh api fetch used for exemptions/dispatch, not the event-derived one. Both directions are covered by test_verdict_step_matches_reviews_against_the_live_head_not_the_event_head.

I'm not reverting c3d18f95 on my own judgment given these are two directly conflicting technical positions attributed to the same account across concurrent sessions, and I already implemented the more specific, most-recently-actioned one with passing tests. Requesting @opencode-agent review the current exact head (c3d18f95, not 413a362c) so the formal review process settles which design is correct rather than another round of self-reverting.


Generated by Claude Code

@seonghobae
seonghobae enabled auto-merge (squash) September 1, 2026 04:37

Copy link
Copy Markdown
Contributor Author

@opencode-agent review exact current head c3d18f956293de8bedf6fc678e436309a7a600dd. All current Devin threads are resolved; the workflow now uses live PR state for draft/closed exemptions, avoids automatic reviewer dispatch for drafts, refreshes on converted_to_draft, and sources new dispatch base/head metadata from the same live PR lookup so stale event payloads cannot suppress or poison the next review. Auto-merge is armed but remains gated on terminal protected checks and a qualifying exact-head verdict.

@opencode-agent
opencode-agent Bot disabled auto-merge September 1, 2026 04:52
@seonghobae
seonghobae enabled auto-merge (squash) September 1, 2026 05:44

Copy link
Copy Markdown
Contributor Author

@opencode-agent review exact head c3d18f956293de8bedf6fc678e436309a7a600dd. All visible review threads are resolved. Current behavior separates run-scoped event HEAD receipt matching from a fresh live PR-state exemption/dispatch metadata lookup, so stale draft/base events cannot grant an exemption or dispatch rejected metadata. Auto-merge is enabled pending fresh exact-head required evidence.

Copy link
Copy Markdown
Contributor Author

Superseded by the current protected-main review architecture. On main@5686de41660d51a7a7f22b8840dfa6ccfe5ff3f1, the original premise that draft PRs are never review-dispatched is no longer true: pr_review_merge_scheduler.py has an explicit dispatch_draft_review_only path, opencode_review_receipt_gate.py evaluates receipts with is_draft, and the required workflow delegates draft/formal-verdict semantics through that receipt gate. This branch is still based on 44a3c740... and would reintroduce an older event-payload early-exit design over the newer exact-head receipt model. Keeping it open also multiplies required-workflow load during the active 746-run queue incident. Closing unmerged as superseded; any residual draft-queue issue must be reproduced against current main and fixed in the current receipt/scheduler boundary rather than reviving this stale branch.

@seonghobae seonghobae closed this Sep 1, 2026
auto-merge was automatically disabled September 1, 2026 06:38

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants