Repository navigation
fix(ci): restore executable hourly governance #36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
seonghobae
wants to merge
19
commits into
main
Choose a base branch
from
fix/hourly-loop-reachable-governance
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 11 commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
d452500
test(ci): define hourly governance repair
seonghobae 51fb2c6
ci: verify and apply hourly loop repair
seonghobae e80f719
ci: allow PR-triggered hourly repair bootstrap
seonghobae c320e7c
test(ci): align hourly repair assertions
seonghobae 3eb3cdb
ci: apply complete hourly workflow test repair
seonghobae 5e32251
ci: export verified hourly repair bundle
seonghobae 2389f2e
ci: stage workflow replacement outside protected path
seonghobae e65f518
fix(ci): stage executable hourly governance
github-actions[bot] a05fe5a
fix(ci): restore executable hourly governance
seonghobae 0d9ec79
test(ci): pin hourly secret boundaries
seonghobae 05f33e3
test(ci): format hourly secret contract
seonghobae 3e3cb6a
fix(ci): enforce secretless deterministic gate
seonghobae d660011
test(ci): align secret boundary expectations
seonghobae ec6a266
chore(ci): retrigger required review after stale change-requests
seonghobae f2bbc04
docs: fix orphaned trust-zones heading
seonghobae d28f92a
Merge origin/main into fix/hourly-loop-reachable-governance (restack …
seonghobae 4c2cfd4
docs(workflow): remove stale repair claims
seonghobae 254977c
test(ci): cover repair ordering and count failure
seonghobae 5c4d13f
fix(ci): serialize repair hold before revalidation
seonghobae File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| # ADR 0006: Fail closed when the central repair workflow is unreachable | ||
|
|
||
| - **Status:** Accepted | ||
| - **Date:** 2026-08-07 | ||
|
|
||
| ## Context | ||
|
|
||
| The hourly RankWeave workflow composed central inspection, review repair, | ||
| revalidation, and local NVIDIA NIM product development. Its review-repair call | ||
| was pinned to a central commit that became unreachable from protected central | ||
| history. GitHub rejected each scheduled caller before creating jobs, disabling | ||
| the whole loop. | ||
|
|
||
| The current protected central repair implementation still uses GitHub Models, | ||
| while a reviewed NVIDIA NIM replacement remains outside protected main. Calling | ||
| either the orphaned SHA, mutable central `main`, or an unmerged branch would | ||
| violate the product's credential and immutable-source boundaries. | ||
|
|
||
| ## Decision | ||
|
|
||
| Keep the two immutable reachable merge-scheduler calls. Replace review repair | ||
| with a local read-only hold job until the protected central NVIDIA NIM repair | ||
| engine is available at a reachable immutable SHA. The hold job may inspect only | ||
| the open-PR count and must not receive mutation, OIDC, provider, or inherited | ||
| secret permissions. | ||
|
|
||
| ## Consequences | ||
|
|
||
| - The hourly workflow executes instead of failing during reusable-workflow | ||
| resolution. | ||
| - PR inspection and revalidation continue each hour. | ||
| - Product development can proceed when all governance jobs succeed and the PR | ||
| queue is empty. | ||
| - Review repair remains unavailable rather than silently routing through an | ||
| unapproved provider or mutable control plane. | ||
| - Re-enabling repair requires a focused PR that pins the protected central | ||
| NVIDIA scheduler and updates tests, operations documentation, and this ADR's | ||
| supersession record. | ||
|
|
||
| ## Diagram | ||
|
|
||
| ```mermaid | ||
| flowchart LR | ||
| S[Hourly schedule] --> I[Immutable central inspection] | ||
| I --> H[Local read-only repair hold] | ||
| H --> R[Immutable central revalidation] | ||
| R -->|PR queue empty| N[NVIDIA NIM product development] | ||
| R -->|PR open| Q[Ordinary review and checks] | ||
| C[Protected central NVIDIA repair] -. future reachable SHA .-> H | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| # Hourly reusable-workflow reachability incident | ||
|
|
||
| - **Date:** 2026-08-07 | ||
| - **Component:** `.github/workflows/hourly-commercialization-loop.yml` | ||
| - **Failure:** scheduled workflow concluded `failure` before GitHub created any | ||
| jobs. | ||
|
|
||
| ## Root cause | ||
|
|
||
| RankWeave pinned the central review-fix reusable workflow to commit | ||
| `21397126d708d2d536ccc1d68b0d333653ce9315`. That commit later diverged from the | ||
| protected central history, so the caller could no longer resolve the reusable | ||
| workflow. Recent failed runs contained zero jobs, while the last successful | ||
| hourly run used the same RankWeave caller before the central ref became | ||
| unreachable. | ||
|
|
||
| ## Remediation | ||
|
|
||
| The local hourly workflow now retains its reachable immutable merge-scheduler | ||
| calls and replaces the unavailable repair call with a read-only local hold job. | ||
| The bridge checks whether an open PR exists and records the fail-closed repair | ||
| state, but it has no write, OIDC, issue, provider, or model credential. It does | ||
| not copy the repair engine and does not fall back to GitHub Models. | ||
|
|
||
| The central repair call may return only after a protected central NVIDIA | ||
| NIM/OpenCode scheduler has merged and RankWeave pins its reachable immutable | ||
| commit. The existing independent review workflows and their credentials remain | ||
| unchanged. | ||
|
|
||
| ## Verification | ||
|
|
||
| - Contract tests reject any `pr-review-fix-scheduler.yml@...` reference in the | ||
| temporary bridge state. | ||
| - Contract tests require two immutable merge-scheduler calls. | ||
| - Contract tests require the bridge to remain local, read-only, secret-free, | ||
| provider-neutral, and bounded. | ||
| - Full Python 3.10-3.13 CI, package smoke, Security Scan, and SAST must pass on | ||
| the exact PR head before merge. | ||
|
|
||
| ## Rollback | ||
|
|
||
| Restore a central review-repair call only with a protected, reachable, reviewed | ||
| commit SHA whose workflow uses NVIDIA NIM/OpenCode and preserves the existing | ||
| review-agent credential boundary. Never restore the orphaned SHA or substitute | ||
| a mutable branch. | ||
|
|
||
| ## References | ||
|
|
||
| GitHub. (2026). *Reusing workflow configurations*. GitHub Docs. | ||
| https://docs.github.com/en/actions/reference/workflows-and-actions/reusing-workflow-configurations | ||
|
|
||
| GitHub. (2026). *GITHUB_TOKEN*. GitHub Docs. | ||
| https://docs.github.com/en/actions/concepts/security/github_token |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| from pathlib import Path | ||
|
|
||
| PROJECT_ROOT = Path(__file__).resolve().parents[1] | ||
| WORKFLOW_PATH = PROJECT_ROOT / ".github/workflows/hourly-commercialization-loop.yml" | ||
|
|
||
|
|
||
| def _workflow_text() -> str: | ||
| return WORKFLOW_PATH.read_text(encoding="utf-8") | ||
|
|
||
|
|
||
| def _job_section(workflow: str, job_name: str, next_job_name: str) -> str: | ||
| start = workflow.index(f" {job_name}:\n") | ||
| end = workflow.index(f" {next_job_name}:\n", start) | ||
| return workflow[start:end] | ||
|
|
||
|
|
||
| def _step_section(workflow: str, step_name: str, next_step_name: str) -> str: | ||
| start = workflow.index(f" - name: {step_name}\n") | ||
| end = workflow.index(f" - name: {next_step_name}\n", start) | ||
| return workflow[start:end] | ||
|
|
||
|
|
||
| def test_merge_governance_does_not_inherit_all_repository_secrets(): | ||
| workflow = _workflow_text() | ||
| inspect = _job_section(workflow, "inspect-pr-queue", "repair-review-feedback") | ||
| revalidate = _job_section( | ||
| workflow, | ||
| "revalidate-pr-queue", | ||
| "develop-next-product-gap", | ||
| ) | ||
|
|
||
| # The pinned merge scheduler has same-repository GITHUB_TOKEN authority and | ||
| # an OIDC app-token path. RankWeave must not forward every repository secret | ||
| # merely to call that reusable governance workflow. | ||
| assert "secrets: inherit" not in inspect | ||
| assert "secrets: inherit" not in revalidate | ||
|
|
||
|
|
||
| def test_nvidia_secret_materialization_follows_the_deterministic_queue_gate(): | ||
| workflow = _workflow_text() | ||
| gate = _step_section( | ||
| workflow, | ||
| "Determine whether product development may start", | ||
| "Check out the current base without persisted credentials", | ||
| ) | ||
| red_authoring = _step_section( | ||
| workflow, | ||
| "Author one design and failing regression test", | ||
| "Verify test-only scope and observe the red state", | ||
| ) | ||
|
|
||
| # An open PR is a deterministic stop and must be decided before any model | ||
| # credential is materialized. NVIDIA credentials belong only to the actual | ||
| # model-backed authoring step after that decision. | ||
| assert "/pulls?state=open&per_page=1" in gate | ||
| assert "NVIDIA_NIM_API_KEY" not in gate | ||
| assert "NVIDIA_API_KEY" not in gate | ||
| assert "NVIDIA_API_KEY: ${{ secrets.NVIDIA_NIM_API_KEY }}" in red_authoring | ||
| assert 'if [ -z "${NVIDIA_API_KEY:-}" ]; then' in red_authoring |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.