Skip to content
Draft
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
1d9c70b
fix(codeql): keep a clean dispatch scan when status publish 403s
seonghobae Sep 8, 2026
6c4ed67
fix(codeql): dispatch remaining languages on workflow reruns
seonghobae Sep 8, 2026
c92e336
test(codeql): require paginated dispatch evidence lookup
seonghobae Sep 8, 2026
3cefd2c
test(codeql): require paginated dispatch job lookup
seonghobae Sep 8, 2026
d51da47
fix(codeql): paginate exact dispatch evidence
seonghobae Sep 8, 2026
26e4e80
test(codeql): parse paginated gh endpoint options
seonghobae Sep 8, 2026
c99d49a
style(codeql): normalize pagination fixture spacing
seonghobae Sep 8, 2026
e715a5e
test(codeql): match paginated empty dispatch fixtures
seonghobae Sep 8, 2026
fd76b88
fix(codeql): 언어별 재시작 경합과 재분석 반복 방지
seonghobae Sep 8, 2026
fc78e16
Merge commit 'e715a5e5e5235130cd8d6054f14088cd2c6cad6f' into codex/co…
seonghobae Sep 8, 2026
2268986
fix(codeql): restack coordinated failed-job recovery
seonghobae Sep 8, 2026
b97d982
test(codeql): bind coordinator fallback to exact dispatch identity
seonghobae Sep 8, 2026
78aca1d
test(codeql): align coordinator fallback fixture with required run
seonghobae Sep 8, 2026
868a208
fix(codeql): bind coordinator fallback to exact dispatch run
seonghobae Sep 8, 2026
d1affc0
docs(gap): record exact CodeQL coordinator identity repair
seonghobae Sep 8, 2026
c19d31f
docs(actions): distinguish workflow counts from job capacity
seonghobae Sep 8, 2026
d66f1d5
test(codeql): execute coordinator identity contract
seonghobae Sep 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 52 additions & 1 deletion .github/workflows/codeql-pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,38 @@ jobs:
exit 0
;;
esac

expected_title="CodeQL Scan Dispatch ${TARGET_REPOSITORY}#${PR_NUMBER}@${PR_HEAD_SHA}"
expected_job="CodeQL dispatch scan (${LANGUAGE})"
runs_json="$(gh api --paginate --slurp "repos/ContextualWisdomLab/.github/actions/workflows/codeql-scan-dispatch.yml/runs")"
run_id="$(printf '%s' "$runs_json" | jq -r --arg title "$expected_title" --arg path ".github/workflows/codeql-scan-dispatch.yml" '
[
.[] | .workflow_runs[]
| select(.path == $path)
| select(.event == "repository_dispatch")
| select(.status == "completed" or .status == "in_progress")
| select(.display_title == $title or .name == $title)
]
| first
| .id // empty
')"
if [[ "$run_id" =~ ^[1-9][0-9]*$ ]]; then
jobs_json="$(gh api --paginate --slurp "repos/ContextualWisdomLab/.github/actions/runs/${run_id}/jobs")"
job_conclusion="$(printf '%s' "$jobs_json" | jq -r --arg name "$expected_job" '
[.[] | .jobs[]] as $jobs
| select([$jobs[] | select(.name == "validate-dispatch" and .status == "completed" and .conclusion == "success")] | length == 1)
| [$jobs[] | select(.name == $name and .status == "completed")]
| if length == 1 then .[0].conclusion else empty end
')"
case "$job_conclusion" in
success|failure)
echo "verdict=${job_conclusion}" >>"$GITHUB_OUTPUT"
echo "Found completed CodeQL dispatch scan job for ${LANGUAGE}: ${job_conclusion}."
exit 0
;;
esac
fi

if [ "$RUN_ATTEMPT" != "1" ]; then
echo "::error::Exact CodeQL job was rerun without an authenticated terminal verdict."
exit 1
Expand Down Expand Up @@ -251,7 +283,6 @@ jobs:
always()
&& github.event.action != 'closed'
&& github.event.pull_request.state != 'closed'
&& github.run_attempt == 1
&& needs.detect-languages.result == 'success'
&& needs.detect-languages.outputs.code == 'true'
runs-on: ubuntu-24.04
Expand Down Expand Up @@ -325,6 +356,18 @@ jobs:
done < <(printf '%s' "$include_json" | jq -c '.[]')

statuses="$(gh api "repos/${TARGET_REPOSITORY}/commits/${PR_HEAD_SHA}/statuses")"
expected_title="CodeQL Scan Dispatch ${TARGET_REPOSITORY}#${PR_NUMBER}@${PR_HEAD_SHA}"
dispatch_runs="$(gh api --paginate --slurp "repos/ContextualWisdomLab/.github/actions/workflows/codeql-scan-dispatch.yml/runs")"
dispatch_run_id="$(printf '%s' "$dispatch_runs" | jq -r --arg title "$expected_title" '
[.[] | .workflow_runs[]
| select(.path == ".github/workflows/codeql-scan-dispatch.yml" and .event == "repository_dispatch")
| select(.status == "completed" or .status == "in_progress")
| select(.display_title == $title or .name == $title)] | first | .id // empty
')"
dispatch_jobs='[]'
if [[ "$dispatch_run_id" =~ ^[1-9][0-9]*$ ]]; then
dispatch_jobs="$(gh api --paginate --slurp "repos/ContextualWisdomLab/.github/actions/runs/${dispatch_run_id}/jobs")"
fi
pending_matrix='[]'
while IFS= read -r entry; do
language="$(printf '%s' "$entry" | jq -r '.language // empty')"
Expand All @@ -339,6 +382,14 @@ jobs:
]
| first // {} | .state // empty
')"
if ! [[ "$verdict_state" =~ ^(success|failure|error)$ ]]; then
verdict_state="$(printf '%s' "$dispatch_jobs" | jq -r --arg name "CodeQL dispatch scan (${language})" '
[.[] | .jobs[]] as $jobs
| select([$jobs[] | select(.name == "validate-dispatch" and .status == "completed" and .conclusion == "success")] | length == 1)
| [$jobs[] | select(.name == $name and .status == "completed")]
| if length == 1 then .[0].conclusion else empty end
')"
fi
case "$verdict_state" in
success|failure|error)
echo "Found authenticated current-head CodeQL verdict for ${language}: ${verdict_state}."
Expand Down
60 changes: 38 additions & 22 deletions .github/workflows/codeql-scan-dispatch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,7 @@ jobs:
runs-on: ubuntu-24.04
timeout-minutes: 30
permissions:
actions: write
actions: read
contents: read
security-events: read
id-token: write
Expand Down Expand Up @@ -503,13 +503,31 @@ jobs:
exit 0
fi

if [ "$GATE_OUTCOME" = "success" ]; then
echo "::notice::Could not publish the CodeQL dispatch status after all configured credentials failed. The completed dispatch scan job remains the evidence for this head."
exit 0
fi

echo "::error::Could not publish the CodeQL dispatch status after all configured credentials failed; the exact required job will remain failed and will not be woken with stale or missing evidence."
exit 1

wake-required:
name: Wake verified CodeQL required jobs
needs: [validate-dispatch, scan]
if: >-
always()
&& needs.validate-dispatch.result == 'success'
&& needs.scan.result != 'cancelled'
&& needs.scan.result != 'skipped'
runs-on: ubuntu-24.04
timeout-minutes: 8
permissions:
actions: write
contents: read
steps:
- name: Wake exact CodeQL required job
if: >-
always()
&& steps.publish_status.outcome == 'success'
&& needs.validate-dispatch.outputs.target_repository != ''
&& needs.validate-dispatch.outputs.pr_number != ''
&& needs.validate-dispatch.outputs.head_sha != ''
Expand All @@ -522,21 +540,14 @@ jobs:
HEAD_SHA: ${{ needs.validate-dispatch.outputs.head_sha }}
REQUIRED_RUN_ID: ${{ needs.validate-dispatch.outputs.required_run_id }}
REQUIRED_JOBS: ${{ needs.validate-dispatch.outputs.required_jobs }}
REQUIRED_LANGUAGE: ${{ matrix.language }}
WAKE_TOKEN_SOURCE: ${{ needs.validate-dispatch.outputs.target_repository == github.repository && 'github-token' || secrets.PR_REVIEW_MERGE_TOKEN != '' && 'PR_REVIEW_MERGE_TOKEN' || secrets.OPENCODE_APPROVE_TOKEN != '' && 'OPENCODE_APPROVE_TOKEN' || 'unavailable' }}
run: |
set -euo pipefail
if [ -z "${GH_TOKEN:-}" ] || [ "$WAKE_TOKEN_SOURCE" = "unavailable" ]; then
echo "::error::Actions-capable CodeQL wake credential is unavailable."
exit 1
fi
REQUIRED_JOB_ID="$(printf '%s' "$REQUIRED_JOBS" | jq -r --arg lang "$REQUIRED_LANGUAGE" '
[.[] | select(.language == $lang) | .job_id | tostring]
| if length == 1 and (.[0] | test("^[1-9][0-9]*$")) then .[0] else empty end
')"
if ! [[ "$REQUIRED_RUN_ID" =~ ^[1-9][0-9]*$ ]] ||
! [[ "$REQUIRED_JOB_ID" =~ ^[1-9][0-9]*$ ]] ||
! [[ "$REQUIRED_LANGUAGE" =~ ^[a-z0-9-]+$ ]]; then
if ! [[ "$REQUIRED_RUN_ID" =~ ^[1-9][0-9]*$ ]]; then
echo "::error::CodeQL wake identity is non-canonical."
exit 1
fi
Expand All @@ -555,23 +566,28 @@ jobs:
| select(.event == "pull_request")
| select(.path == ".github/workflows/codeql-pr.yml")
| select(.head_sha == $head)
| .id // empty
')"
expected_name="CodeQL compatibility analysis (${REQUIRED_LANGUAGE})"
job="$(gh api "repos/${TARGET_REPOSITORY}/actions/jobs/${REQUIRED_JOB_ID}")"
job_identity="$(printf '%s' "$job" | jq -r --arg head "$HEAD_SHA" --arg name "$expected_name" --argjson run_id "$REQUIRED_RUN_ID" --argjson job_id "$REQUIRED_JOB_ID" '
select(.id == $job_id)
| select(.run_id == $run_id)
| select(.head_sha == $head)
| select(.name == $name)
| select(.status == "completed" and .conclusion == "failure")
| .id // empty
')"
jobs="$(gh api --paginate --slurp "repos/${TARGET_REPOSITORY}/actions/runs/${REQUIRED_RUN_ID}/jobs")"
jobs_verified="$(printf '%s' "$jobs" | jq -r --arg head "$HEAD_SHA" --argjson run_id "$REQUIRED_RUN_ID" --argjson required "$REQUIRED_JOBS" '
[.[] | .jobs[]] as $jobs
| [$jobs[] | select(.conclusion == "failure")] as $failed
| ($required | length > 0)
and ($failed | length > 0)
and ($jobs | all(.status == "completed"))
and ($required | all(. as $expected |
[$jobs[] | select(.id == $expected.job_id)
| select(.run_id == $run_id and .head_sha == $head)
| select(.name == ("CodeQL compatibility analysis (" + $expected.language + ")"))
| select(.conclusion == "failure")] | length == 1))
and (($failed | map(.id) | sort) == ($required | map(.job_id) | sort))
')"
if [ "$run_identity" != "$REQUIRED_RUN_ID" ] ||
[ "$job_identity" != "$REQUIRED_JOB_ID" ]; then
[ "$jobs_verified" != "true" ]; then
echo "::error::CodeQL wake rejected missing or ambiguous exact run/job identity."
exit 1
fi

gh api -X POST "repos/${TARGET_REPOSITORY}/actions/jobs/${REQUIRED_JOB_ID}/rerun" >/dev/null
echo "Re-ran exact failed CodeQL job ${REQUIRED_JOB_ID} for ${REQUIRED_LANGUAGE} on ${HEAD_SHA}."
gh api -X POST "repos/${TARGET_REPOSITORY}/actions/runs/${REQUIRED_RUN_ID}/rerun-failed-jobs" >/dev/null
echo "Re-ran verified failed CodeQL jobs once for run ${REQUIRED_RUN_ID} on ${HEAD_SHA}."
14 changes: 14 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -212,3 +212,17 @@ them alone proves succession.
variable in CI, so a failure class exists that cannot reproduce locally. Before calling a
scheduler change clean, run the affected tests both ways, including
`GITHUB_ACTIONS=true python3 -m pytest <paths>`.
- CodeQL wake test doubles must reproduce GitHub rejecting a job rerun while
its containing run is already running. A fake POST that always succeeds hides
the language-shard race observed in dispatch run `34178442472`. Keep scan,
status publication, authenticated verdict consumption, and required-job
recovery as separate outcomes; a published `success` from an unaccepted
creator is not a passing required check. Do not mask this with blanket 403
suppression, sleep polling, or a wider unauthenticated creator allowlist.
- Finish the language matrix before one trusted coordinator requests recovery.
Before a native failed-jobs rerun, verify the current PR head, completed failed
run, and the complete failed-job set against the supplied CodeQL identities.
Keep verdict readers and dispatch admission aligned: completed validated scan
jobs must stop redispatch after both success and real findings, even while
the recovery coordinator is still finishing. Otherwise recovery starts a new
scan loop or waits for its own containing workflow to finish.
55 changes: 55 additions & 0 deletions docs/doctoring/codeql-matrix-wake-race.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# CodeQL matrix recovery race

Status: local implementation under verification; not merged or deployed.

## Evidence and failure boundary

On 2026-09-08, `.github` PR #2029 head
`eb79481bc1696c63273b6c2ca22b5e34f68d0208` produced two successful scan
statuses in dispatch run `34178442472`. Python job `101912523347` then failed
at the wake step with `The workflow run containing this job is already running
(HTTP 403)`. The actions shard had already requested recovery of required run
`34177963535`. This was not a CodeQL finding or a status publication denial.
Both published statuses were authored by `github-actions[bot]`, which the
required consumer deliberately does not trust as the OpenCode app identity.

The existing test double accepted every POST. On base
`c99d49a86a2728d136d9e3a34a6b82a2c0e84a90`, all 24 dispatch tests passed.
Reproducing GitHub's observed active-run rejection made the parallel-language
test fail while completed-run recovery still passed (1 failed, 1 passed).

## Proposed repair and alternatives

One coordinator follows the complete scan matrix and requests one native
failed-jobs rerun. It validates the live open PR head, exact workflow/run,
terminal run state, and equality of the entire failed-job set with the supplied
CodeQL job identities. An unrelated failure, stale identity, or active run
fails closed without a POST. Scan jobs retain read-only Actions access;
recovery alone needs write access. No polling or extra model invocation is added.

The consumer can read a completed scan job only after the same dispatch run's
validation job succeeds. The coordinator may still be finishing; requiring
the entire dispatch run to finish would introduce a race with the jobs it
just restarted. Admission uses the same terminal-evidence conditions to stop
rescanning both clean results and real findings. Findings remain failed checks.

Rejected alternatives: ignoring 403 loses recovery; repeating per-language
requests races again; sleeping occupies runners without establishing identity;
trusting every status creator weakens the verification boundary. A separate
completion-event workflow adds another event and evidence-transfer contract.

## Verification still required

Run both CodeQL contract modules, broader workflow admission/image contracts,
and current-head hosted checks. Inspect the rendered guidance. Prove one
recovery request for a real multilingual PR and terminal required checks before
claiming rollout. Local fake APIs cannot establish hosted convergence or relief
of the organization-wide 60-job ceiling.

## Reference

GitHub. (n.d.). *REST API endpoints for workflow runs*. Retrieved September 8,
2026, from https://docs.github.com/en/rest/actions/workflow-runs

The documented failed-jobs endpoint restarts failed jobs and their dependent
jobs; therefore the full failed-job set must be validated before calling it.
Loading
Loading