Skip to content

feat: expand autonomous fleet operations and delivery safeguards - #2038

Closed
quinnbot-ai wants to merge 4 commits into
kunchenguid:mainfrom
quinnbot-ai:fm/fm-gh-ci-fallback-signature
Closed

quinnbot-ai wants to merge 4 commits into
kunchenguid:mainfrom
quinnbot-ai:fm/fm-gh-ci-fallback-signature

Conversation

@quinnbot-ai

Copy link
Copy Markdown

Intent

Fix bin/fm-gh-ci-fallback.sh so it recognizes GitHub's CURRENT permission-denial signature for gh pr checks under a fine-grained personal access token.

The bug (live, blocking a real delivery): the no-mistakes daemon's gh pr checks call under a fine-grained PAT now exits 1 with 'GraphQL: Resource not accessible by personal access token (node.statusCheckRollup.nodes.0.commit.statusCheckRollup.contexts.nodes.0)'. The bounded CI fallback in bin/fm-gh-ci-fallback.sh activated only on a denial signature ending directly after 'statusCheckRollup)'. GitHub appended the deeper '.contexts.nodes.0' path segment, so the fallback never activated and an active no-mistakes run loops on the denial forever even when the exact head's workflow is green.

Required change, as accepted:

  • Generalize the denial-signature recognition to match GitHub's permission-denial for statusCheckRollup reads regardless of the trailing property path, matching the stable prefix 'Resource not accessible by personal access token' combined with a statusCheckRollup path component, in whatever style the existing matcher used, WITHOUT loosening it into matching unrelated GraphQL errors.
  • Preserve the fallback's exact supported argument shapes and its exact-head workflow verification unchanged.
  • Add regression coverage in the colocated test file: at minimum the old signature (still recognized), the new '.contexts.nodes.0' signature (now recognized), and a clearly unrelated GraphQL error (still NOT recognized, fallback stays inactive).
  • This is firstmate shared tracked material, so .agents/skills/firstmate-coding-guidelines/SKILL.md applies.

Acceptance criteria: with the new signature the fallback activates and the CI verdict comes from the exact-head workflow verification path exactly as it did for the old signature; unrelated errors still do not activate the fallback; tests colocated and green; shellcheck-clean per repo conventions.

Decisions and tradeoffs made while implementing, which a reviewer reading only the diff would not know:

  • The matcher was deliberately changed from a fixed-string grep -Fq to an ERE grep -Eq, not to a substring or looser match. The pattern requires GitHub's exact denial sentence AND a well-formed dotted GraphQL path in which 'statusCheckRollup' appears as a WHOLE component: 'GraphQL: Resource not accessible by personal access token (([A-Za-z0-9_]+.)statusCheckRollup(.[A-Za-z0-9_]+))'. Component-exactness is intentional: a path like 'node.statusCheckRollupSummary.nodes.0' must NOT activate the fallback, so that a future unrelated field whose name merely starts with those characters cannot be reinterpreted as CI state.
  • No behavior other than signature recognition was touched: fallback_shape_parse, the two supported argument vectors, the exact-head PR lookup, the workflow-runs pagination, the jq bucket mapping, and the post-pagination head-drift recheck are all unchanged, because narrowing the blast radius of this fix was more important than any cleanup.
  • Regression coverage was added to the existing colocated tests/fm-gh-shim.test.sh rather than a new runner, per repo test conventions: a new fake-gh error case '403-deep-path' emitting the live deeper denial, a new test asserting it reaches the same green exact-head verdict and calls only the exact head_sha Actions endpoint, and a new '403-lookalike' near-miss case folded into the existing unrelated-failure loop that asserts the lookalike path is replayed unchanged and never reaches the fallback API. The old signature stays covered by all the pre-existing CI cases, which continue to use it as the fake's default error.
  • The new test was verified non-vacuous: with the old literal matcher restored, the deep-path test fails; with the new matcher it passes.
  • Two maintained prose surfaces were corrected to match current behavior rather than left to rot: docs/no-mistakes-pr-credential.md described the trigger as the exact check-runs 403 signature, and docs/verification/gh-pr-credential.md is a maintainer-verification record whose CI case count and recorded test roster had to match the actual test output, which was re-run and diffed to confirm.
  • Verification run locally: tests/fm-gh-shim.test.sh green (22 cases), tests/fm-pr-merge.test.sh green, bin/fm-lint.sh clean on the pinned shellcheck 0.11.0, bin/fm-doc-audience-check.sh clean.

What Changed

  • Add remote secondmate lifecycle support, durable link/process-event/public-followup flows, task trace propagation, and capacity-aware fleet refill supervision.
  • Expand Calm presentation across Pi and Claude while hardening Herdr, tmux, and Zellij launch, liveness, locking, teardown, and recovery behavior.
  • Add declarative test gates, durable test progress and stock-Bash CI coverage, plus configurable GitHub PR credentials and exact-head workflow fallback for fine-grained PAT status-check denials.

Risk Assessment

✅ Low: The focused matcher change safely recognizes whole-component statusCheckRollup denial paths while preserving argument-shape and exact-head fallback boundaries, with appropriate positive and negative regression coverage.

Testing

The supplied baseline reported shim, merge, lint, and documentation checks green; this phase independently reran the targeted shim suite and direct CLI scenarios, confirming both valid denial signatures return the green exact-head workflow verdict while the unrelated lookalike is replayed unchanged without fallback API calls.

Evidence: Direct fallback CLI transcript
COMMAND: gh pr checks 7 --repo o/r --json name,state,bucket,completedAt

CASE: 403
EXIT: 0
OUTPUT:
[{"name":"exact-head","state":"success","bucket":"pass","completedAt":"2026-08-07T12:34:56Z","link":"https://github.com/o/r/actions/runs/123"}]
API TRACE:
argv:pr checks 7 --repo o/r --json name,state,bucket,completedAt
argv:api -X GET repos/o/r/pulls/7 --jq .head.sha
argv:api -X GET repos/o/r/actions/runs?head_sha=9999999999999999999999999999999999999999&per_page=100 --paginate --slurp --jq [
argv:api -X GET repos/o/r/pulls/7 --jq .head.sha

CASE: 403-deep-path
EXIT: 0
OUTPUT:
[{"name":"exact-head","state":"success","bucket":"pass","completedAt":"2026-08-07T12:34:56Z","link":"https://github.com/o/r/actions/runs/123"}]
API TRACE:
argv:pr checks 7 --repo o/r --json name,state,bucket,completedAt
argv:api -X GET repos/o/r/pulls/7 --jq .head.sha
argv:api -X GET repos/o/r/actions/runs?head_sha=9999999999999999999999999999999999999999&per_page=100 --paginate --slurp --jq [
argv:api -X GET repos/o/r/pulls/7 --jq .head.sha

CASE: 403-lookalike
EXIT: 1
OUTPUT:
GraphQL: Resource not accessible by personal access token (node.statusCheckRollupSummary.nodes.0)
API TRACE:
argv:pr checks 7 --repo o/r --json name,state,bucket,completedAt
Evidence: Colocated behavioral suite
ok - gh pr create without --repo injects its checkout origin into the credential route
ok - repository-like option values cannot suppress origin targeting
ok - caller-supplied --repo and -R always win over the checkout origin
ok - credential-routed pr edit without --repo refuses when origin is unavailable
ok - a check-runs 403 reaches a green exact-head workflow verdict without the privileged token
ok - a denial naming statusCheckRollup deeper in its path still reaches the exact-head verdict
ok - a check-runs 403 reaches a red exact-head workflow verdict
ok - zero exact-head workflow runs remain explicitly pending
ok - paginated exact-head runs map cancellation, skipping, and pending buckets
ok - a moving PR head cannot emit a stale workflow verdict
ok - unrelated check failures are preserved without an API fallback
ok - unsupported gh pr checks shapes exec the real gh directly
ok - pr list, pr view, and api pass through the shim without the privileged credential
ok - fm-gh.sh with no config/gh-credential execs the command unchanged
ok - configured fm-gh.sh exposes only the prefix-injected GitHub token
ok - fm-gh.sh reads only the first non-empty, non-comment, trimmed prefix line
ok - fm-gh.sh ignores credential comments after trimming whitespace
ok - the shim routes at most once even when the credential re-resolves gh from PATH
ok - fm-gh.sh runs both the unconfigured and configured paths under bash 3.2
ok - the installer creates, verifies, reports precedence for, and removes the shim
ok - the installer refuses to overwrite or remove a gh it does not own
ok - the installer refuses a foreign gh symlink and leaves it intact

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

⚠️ **Rebase** - 1 warning

Push main to origin, or rebase your branch onto origin/main, before gating.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

  • tests/fm-gh-shim.test.sh
  • Manual gh pr checks 7 --repo o/r --json name,state,bucket,completedAt scenarios for the old denial, current deeper denial, and statusCheckRollupSummary near-miss, recording exit status, JSON output, and API trace.
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

quinnbot-ai and others added 4 commits August 9, 2026 17:00
#97)

* feat(gh): route pipeline PR creation through a configurable credential

The no-mistakes pipeline opens pull requests by shelling out to `gh`, and a
GitHub fine-grained personal access token is forbidden from the
createPullRequest mutation, so an ambient token of that class breaks the PR
step at the end of an otherwise green run while every earlier step succeeds.

Investigation of the installed v1.41.2 binary and upstream source at release
1.46.0 found no supported configuration seam: the GitHub provider executes
`gh` by bare name with no binary-path or credential knob, while Bitbucket
Cloud reads its credentials from named environment variables. The step
execution layer already honors a step-scoped PATH and credential environment
through StepContext.Env, but that field is declared test-only and the
production executor never populates it.

Add the firstmate-side mitigation instead of patching the tool:

- fm-gh.sh runs one command with the credential prefix from
  config/gh-credential, and execs unchanged when unconfigured.
- fm-gh-shim.sh routes only `pr create` and `pr edit` through that wrapper
  and delegates everything else to the real gh.
- fm-gh-shim-install.sh installs, removes, and verifies PATH precedence, and
  is never run automatically.

Interception must sit on the daemon's PATH because the PR step runs in the
daemon, so it cannot be scoped to a task worktree; that limit is documented
rather than papered over. docs/proposals/nm-pr-step-interception.md specifies
the upstream config seam that would retire the shim.

The wrapper expands its prefix as ${PREFIX[@]+"${PREFIX[@]}"} because bash
3.2, the system bash macOS ships, rejects an empty array expansion under
`set -u` and would otherwise break every unconfigured home.

* no-mistakes(review): Harden PR credential routing and shim ownership

* no-mistakes(document): Correct PR credential routing documentation

* test(gh): create the fixture root before normalizing it

The gh shim suite resolves its fixture root with `cd` so its installer cases can
compare against the installer's own normalized output. A cleanup trap registered
inside the `fm_test_tmproot` command substitution can remove that directory
before it is ever used, which made the normalizing `cd` fail and collapse every
fixture path to the filesystem root.

Create the directory before normalizing so the suite holds regardless of when the
cleanup trap runs.

---------

Co-authored-by: QuinnBot <quinnbot@proton.me>
* fix(ci): fall back to exact-head workflow runs

* no-mistakes(review): Harden exact-head CI fallback guarantees

* no-mistakes(review): Narrow CI fallback to exact monitor failures

* no-mistakes(review): Match observed statusCheckRollup permission denial

* no-mistakes(document): Refresh GitHub shim verification evidence

---------

Co-authored-by: QuinnBot <quinnbot@proton.me>
* fix(gh): pin shim PR target to origin

* no-mistakes(review): Fix gh shim repository option parsing

* no-mistakes(review): Fix long option repository flag confusion

---------

Co-authored-by: QuinnBot <quinnbot@proton.me>
…llback

GitHub now reports the fine-grained personal-token denial for gh pr checks
with a deeper GraphQL error path (.contexts.nodes.0 appended), so the exact
literal signature match never activated the bounded CI fallback and an active
no-mistakes run looped on the denial even with a green exact-head workflow.

Match the denial sentence plus a whole statusCheckRollup path component at any
depth instead. Denials for other APIs, and paths that only begin with those
characters, still replay unchanged. Supported argument shapes and exact-head
workflow verification are untouched.
@quinnbot-ai
quinnbot-ai force-pushed the fm/fm-gh-ci-fallback-signature branch from 82a6389 to bc0dc7e Compare August 10, 2026 00:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant