ci(#6512): fix validate-agents caller permissions and tag the resolved SHA

[fullsend-ai-coder]

Summary

Maintainer takeover of the agent-authored PR. The original scope (Renovate tracking of the agents gate pin) implemented an instruction that #6512 has since rejected; this PR now carries the actual release fixes, which need workflows permission the code agent doesn't have.

Fixes the two release.yml defects behind the failed v0.37.0 release (startup_failure, run 32615313246) and keeps the drift check as a live, non-blocking release signal.

Changes

release.yml

• validate-agents: grant pull-requests: write + checks: read and document the called workflow's permission contract. GitHub validates a called workflow at parse time and callers can only be downgraded — the missing pull-requests: write (declared by the gate's pull_request_target-only gate job) failed the whole run before any job started. checks: read is pre-granted so a future pin bump to agents main (whose functional-tests-complete declares it) cannot reintroduce the startup failure.
• New resolve-agents job resolves agents main exactly once at release start; tag-agents now tags that SHA instead of re-resolving main at tag time. Closes the fullsend-side resolve-twice race — anything merged into agents mid-release can no longer be tagged unvalidated.
• The drift check runs in resolve-agents as continue-on-error — a stale gate pin annotates every release loudly instead of failing it (the pin bump itself stays a maintainer decision, release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512).

Drift check (scripts/check-agents-gate-pin.sh + test, wired into make script-test)

• Relocated from .github/scripts/ → scripts/: everything under .github/scripts/ is vendored into downstream --vendor installs and fetched into every agent-run workspace, and this fullsend-release-specific check can never work in a consumer repo. The vendormanifest.go registration is reverted accordingly.
• Tightened the pin match to full 40-char SHAs; mismatch message no longer claims "N commits behind" as the primary fact (a non-ancestor pin reports ahead_by: 0).

Reverted

• All renovate.json changes: the customManager re-implemented the lockstep design release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512 rejects, and narrowing the fullsend-ai/ ignore rule would newly expose the pin to the native github-actions manager (which does scan root workflow files — verified via renovate-bot commits to runner-image.yml/lint.yml after the scaffold managerFilePatterns was added).

Vendored .github/scripts/ ship-set fixed (allowlist + prune)

Auditing the vendoring side effect surfaced a broader defect: isVendoredDefaultsInfra shipped everything under .github/scripts/ to consumer repos by path prefix, so repo-local CI tooling entered the user ship-set silently (redact-behaviour-artifacts.sh + both *-test.sh files were already shipping; the drift check nearly followed).

• Allowlist inversion (vendorcontent.go): only the four scripts users actually execute ship — check-fix-eligibility.sh (run by reusable-fix/dispatch) and install-podman.sh / install-openshell.sh / openshell-version.sh (run by the root composite action, invoked as ./.defaults/ from every reusable workflow). The list==walk invariant test now enforces the allowlist instead of transcribing the directory. All four are listed because consumer repos execute them; separately, openshell-version.sh / install-openshell.sh cannot move from their paths regardless, because the agents functional-tests gate, hack/gitlab-runner-vm, and the renovate updater read them from a fullsend checkout (path contract, documented in the code).
• Relocated redact-behaviour-artifacts.sh + test to scripts/ (consumers: fullsend's own e2e.yml + Makefile only); updated the base-branch sparse checkout, relevance regex, paths: trigger and docs.
• Pruning on re-vendor (prepareVendorFiles — the chokepoint every vendor commit path collects through, per Grok review of this PR: acquireAndVendor, the combined scaffold+vendor collect func, and appendVendorTreeFiles all receive the delete entries): previously, files recorded in the old vendor manifest but absent from the new set became untracked orphans in consumer repos — the overwritten manifest stopped tracking them, so even uninstall couldn't remove them. Re-vendoring now diffs the committed manifest against the new set and deletes de-listed files in the same commit, so existing vendored repos self-clean on their next CLI bump. A present-but-invalid manifest fails the vendor step instead of silently orphaning; a missing manifest (first install) passes through.
• Full ship-set audit, two more prefix leaks closed (same class):
  • .github/actions/ shipped wholesale. check-e2e-authorization has no consumer outside fullsend's own e2e.yml/functional-tests.yml and executes scripts/check-e2e-authorization.sh, which does not ship — consumers received an unused, broken-if-invoked action. Actions now ship by explicit allowlist (the five the vendored reusable workflows execute from ./.defaults/); the re-vendor prune removes the orphan from existing repos.
  • The layered scripts/ layer shipped its self-tests (pre-fetch-prior-review-test.sh, reconcile-repos-test.sh, resolve-precommit-tools-test.py — referenced only by make script-test). Both layered walks now skip *-test.sh / *-test.py, with a test pinning that runtime scripts still ship and no test file does.
  • Audited clean: root action.yml (references only allowlisted scripts), every path the shipped surface references resolves to shipping content, the layered runtime scripts are all consumer-executed by design, the GitLab scaffold has no scripts, and hack/gitlab-runner-vm (GitLab-forge runner provisioning) copies its two scripts from a checkout with sha256 verification, independent of vendoring.
• Note: the runtime channel (reusable workflows sparse-checkout .github/scripts/ wholesale into the run workspace) is a YAML path pattern and stays prefix-based — relocation, not the Go allowlist, is what removes files from that channel.

Out of scope (agents-repo side, tracked in #6512)

Gate cross-repo checkout ref, workflow_call output of the validated SHA, and the roll-up job's cross-repo checkout — after those land, the pin can move to @main and tag-agents can tag the gate-validated SHA.

Test plan

• bash scripts/check-agents-gate-pin-test.sh — 6/6 pass
• Live run reports the real pin drift (163 behind) with exit 1
• go test ./internal/scaffold/ — vendormanifest invariant holds after revert
• actionlint clean on release.yml and e2e.yml
• go test ./internal/scaffold/ ./internal/cli/ ./internal/layers/ — allowlist invariant + new StaleVendoredPaths test pass
• bash scripts/redact-behaviour-artifacts-test.sh passes from the new location
• TestIsVendoredDefaultsInfra covers the actions allowlist; TestWalkLayeredContent_ExcludesTestFiles pins the layered filter
• Prune wiring exercised via FakeClient (TestAppendStaleVendoredDeletes + TestVendorCommitPathsPruneStaleFiles): Delete entries flow through both real commit paths (combined collect func, appendVendorTreeFiles); missing manifest passes through, invalid manifest errors
• Grok review of the reworked diff: 1 critical (prune not on all commit paths) — fixed; token-asymmetry warning refuted (agents repo is public, verified); pin-staleness warning is the documented release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512 remainder, not new
• Next tag push: validate-agents starts (no startup_failure); resolve-agents and tag-agents log the same agents SHA

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 3:24 AM UTC · Completed 3:41 AM UTC

Commit: 6582af4 · View workflow run →

[fullsend-ai-review]

Review

Findings

Medium

• [protected-path] .github/workflows/release.yml, .github/workflows/e2e.yml, scripts/check-agents-gate-pin.sh, scripts/check-agents-gate-pin-test.sh, scripts/redact-behaviour-artifacts.sh, scripts/redact-behaviour-artifacts-test.sh — PR modifies files under protected paths (.github/, scripts/). The PR links to issue release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512 and the maintainer explicitly authorized the workflow permissions fix, resolve-once tagging, drift-check script relocation, and vendoring ship-set fix. Human approval is always required for protected-path changes, regardless of context.

• [stale-workflow-description] skills/cutting-releases/post-flight.md:38 — Post-flight doc does not account for the new resolve-agents job. Line 38 says "If either job fails" referring only to validate-agents and tag-agents, but resolve-agents is now a third dependency. The gh run view command on line 46 filters with validate-agents|tag-agents, missing resolve-agents. An operator following this runbook would miss a resolve-agents failure.

Low

• [race-condition] .github/workflows/release.yml — resolve-agents and validate-agents run in parallel (both depend only on release). validate-agents tests a pinned SHA, while resolve-agents resolves current agents/main. If agents/main advances, tag-agents tags a tree validate-agents did not test. The PR narrows this window versus the base branch but does not close it — a known trade-off per release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512.

• [error-handling] internal/cli/vendor.go:240 — appendStaleVendoredDeletes treats any error from ReadVendorManifest as fatal, including corrupt manifests. While intentional to avoid silently orphaning de-listed paths, a corrupt manifest in a consumer repo blocks all future vendor installs with no self-heal path. Recovery requires manually deleting the manifest and re-running vendor.

• [workflow-command-injection] scripts/check-agents-gate-pin.sh:66 — The ::notice:: message interpolates PINNED_SHA without ${VAR//::/} sanitization applied elsewhere in the same script. Structurally unexploitable (value constrained to [a-f0-9]{40} by grep), but inconsistent with the script's own sanitization pattern.

• [commit-convention] .github/workflows/release.yml — The PR title uses ci(#6512) but the vendor ship-set changes fix a user-facing defect (stale files persisting in consumer repos). GoReleaser excludes ci-prefixed changes from release notes. Consider splitting into ci(#6512) for workflow fixes and fix(vendor) for ship-set/prune changes.

• [constructor-convention] internal/cli/vendor_test.go:115 — Pre-existing tests TestAcquireAndVendor_ExplicitPath and TestAcquireAndVendor_CheckoutBuild use &forge.FakeClient{} while the rest of the package uses forge.NewFakeClient(). Not a regression, but a missed normalization opportunity while editing the file.

• [stale-workflow-description] skills/cutting-releases/SKILL.md:184 — Agents tagging flow description does not mention the new resolve-agents job. tag-agents now uses a pre-resolved SHA from resolve-agents rather than re-resolving agents/main at tag time.

Previous run

Review

Findings

Medium

• [protected-path] .github/workflows/release.yml, .github/workflows/e2e.yml, scripts/check-agents-gate-pin.sh, scripts/check-agents-gate-pin-test.sh, scripts/redact-behaviour-artifacts.sh, scripts/redact-behaviour-artifacts-test.sh — PR modifies files under protected paths (.github/, scripts/). The PR links to issue release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512 and the maintainer explicitly authorized the workflow permissions fix, resolve-once tagging, drift-check script relocation, and vendoring ship-set fix. Human approval is always required for protected-path changes, regardless of context.

• [commit-convention] .github/workflows/release.yml — The PR title uses ci(#6512) as the conventional commit prefix, but the vendor ship-set changes fix a user-facing defect: consumer repos received files they should not have and de-listed files persisted as untracked orphans. Per COMMITS.md, user-visible bug fixes should use fix, not ci — GoReleaser will exclude ci-prefixed changes from release notes, so users upgrading will not know their vendored repos will self-clean stale files. If the vendor changes remain in this PR, consider splitting the final merge into at least two commits: ci(#6512) for the workflow fixes, fix(vendor) for the ship-set and prune changes.

Low

• [race-condition] .github/workflows/release.yml — resolve-agents and validate-agents run in parallel (both depend only on release). validate-agents tests a pinned SHA, while resolve-agents resolves current agents/main. If agents/main advances, tag-agents tags a tree validate-agents did not test. The PR narrows this window versus the base branch but does not close it — a known trade-off per release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512.

• [workflow-command-injection] scripts/check-agents-gate-pin.sh — The final ::error:: message interpolates PINNED_SHA, AGENTS_MAIN_SHA, and BEHIND_COUNT without the ${VAR//::/} sanitization applied to RELEASE_YML elsewhere in the same script. Exploitation risk is negligible (hex SHA and integer from authenticated API responses, step runs with continue-on-error: true), but the inconsistency is a code-hygiene gap.

• [scope-creep] internal/cli/vendor.go — The vendor ship-set changes (allowlist inversion, prune-on-re-vendor, VendorCollectFunc signature change, layered test-file exclusion) are not in issue release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512's stated scope. The scope expansion is documented and justified in the PR body as discovered during auditing, and the changes are architecturally sound.

Previous run (2)

Review

Findings

Medium

• [protected-path] .github/workflows/release.yml, .github/workflows/e2e.yml, scripts/check-agents-gate-pin.sh, scripts/check-agents-gate-pin-test.sh, scripts/redact-behaviour-artifacts.sh, scripts/redact-behaviour-artifacts-test.sh — PR modifies files under protected paths (.github/, scripts/). The PR links to issue release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512 and the maintainer explicitly authorized the workflow permissions fix, resolve-once tagging, drift-check script relocation, and vendoring ship-set fix. Human approval is always required for protected-path changes, regardless of context.

Low

• [race-condition] .github/workflows/release.yml — resolve-agents and validate-agents run in parallel (both depend only on release). validate-agents tests a pinned SHA, while resolve-agents resolves current agents/main. If agents/main advances, tag-agents tags a tree validate-agents did not test. The PR narrows this window versus the base branch but does not close it — a known trade-off per release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512.

• [workflow-command-injection] scripts/check-agents-gate-pin.sh — The final ::error:: message interpolates PINNED_SHA, AGENTS_MAIN_SHA, and BEHIND_COUNT without the ${VAR//::/} sanitization applied to RELEASE_YML elsewhere in the same script. Both values come from authenticated GitHub API responses (hex SHA and integer) and the step runs with continue-on-error: true, but the inconsistency is a code-hygiene gap.

• [stale-comment] scripts/check-agents-gate-pin-test.sh:5 — Header comment references bash .github/scripts/check-agents-gate-pin-test.sh but the file is at scripts/check-agents-gate-pin-test.sh.

• [set-type-convention] internal/scaffold/vendorcontent.go — vendoredDefaultsScripts and vendoredDefaultsActions use map[string]bool for set membership, while every other set-like map in the scaffold package uses map[string]struct{} (e.g., executableFiles in scaffold.go, seen maps in vendormanifest.go). The map[string]bool idiom exists in other packages but introduces a second pattern within this package.

Previous run (3)

Review

Findings

Medium

• [protected-path] .github/workflows/release.yml, .github/workflows/e2e.yml, scripts/check-agents-gate-pin.sh, scripts/check-agents-gate-pin-test.sh, scripts/redact-behaviour-artifacts.sh, scripts/redact-behaviour-artifacts-test.sh — PR modifies files under protected paths (.github/, scripts/). The PR links to issue release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512 and the maintainer explicitly authorized the workflow permissions fix, resolve-once tagging, drift-check script relocation, and vendoring ship-set fix. Human approval is always required for protected-path changes, regardless of context.

Low

• [permission-elevation] .github/workflows/release.yml — The validate-agents job adds pull-requests: write and checks: read to its permissions block. These are forwarded to the SHA-pinned called workflow (functional-tests.yml@a8566cd5). The inline comment explains why: GitHub validates called-workflow permissions at parse time even for skipped jobs — the missing grants caused the v0.37.0 startup failure (release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512). The workflow triggers only on tag pushes, so pull-requests: write cannot be exercised on a PR event.

• [edge-case] .github/workflows/release.yml — The tag-agents job receives AGENTS_SHA from needs.resolve-agents.outputs.agents_sha via env var without re-validating before passing to gh api. While resolve-agents validates the SHA with ^[a-f0-9]{40}$ upstream, a defensive guard (if [[ -z "${AGENTS_SHA}" ]]) would produce a clear diagnostic if the output mechanism ever fails silently.

• [race-condition] .github/workflows/release.yml — resolve-agents and validate-agents run in parallel (both depend only on release). validate-agents tests a pinned SHA, while resolve-agents resolves current agents/main. If agents/main advances, tag-agents tags a tree validate-agents did not test. The PR narrows this window versus the base branch but does not close it — a known trade-off per release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512.

• [workflow-command-injection] scripts/check-agents-gate-pin.sh — The final ::error:: message interpolates AGENTS_MAIN_SHA and BEHIND_COUNT without the ${VAR//::/} sanitization applied to RELEASE_YML elsewhere in the same script. Both values come from authenticated GitHub API responses (hex SHA and integer) and the step runs with continue-on-error: true, but the inconsistency is a code-hygiene gap.

• [stale-comment] scripts/check-agents-gate-pin-test.sh:5 — Header comment references bash .github/scripts/check-agents-gate-pin-test.sh but the file is at scripts/check-agents-gate-pin-test.sh.

Previous run (4)

Review

Findings

Medium

• [protected-path] .github/workflows/release.yml, scripts/check-agents-gate-pin.sh, scripts/check-agents-gate-pin-test.sh — PR modifies files under protected paths (.github/, scripts/). The PR links to issue release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512 and the maintainer explicitly authorized the workflow permissions fix, resolve-once tagging, and drift-check script relocation. Human approval is always required for protected-path changes, regardless of context.

Low

• [edge-case] .github/workflows/release.yml — The tag-agents job receives AGENTS_SHA from needs.resolve-agents.outputs.agents_sha via env var without re-validating before passing to gh api. While resolve-agents validates the SHA with ^[a-f0-9]{40}$ upstream, a defensive guard (if [[ -z "${AGENTS_SHA}" ]]) would produce a clear diagnostic if the output mechanism ever fails silently.

• [workflow-command-injection] scripts/check-agents-gate-pin.sh — The final ::error:: message interpolates AGENTS_MAIN_SHA and BEHIND_COUNT without the ${VAR//::/} sanitization applied to RELEASE_YML elsewhere in the same script. Both values come from authenticated GitHub API responses (hex SHA and integer) and the step runs with continue-on-error: true, but the inconsistency is a code-hygiene gap.

• [stale-comment] scripts/check-agents-gate-pin-test.sh:4 — Header comment references bash .github/scripts/check-agents-gate-pin-test.sh but the file is at scripts/check-agents-gate-pin-test.sh.

Previous run (5)

Review

Findings

High

• [auto-close-mismatch] renovate.json — The PR body uses Closes #6512, which will auto-close the issue on merge. A maintainer rescoped issue release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512 to include the Renovate and drift-check work this PR delivers, so the PR's scope is authorized. However, issue release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512 still tracks four additional items (caller permissions fix, cross-repo checkout ref, tag-agents re-resolution, roll-up cross-repo fix) that require separate maintainer PRs touching .github/workflows/. Auto-closing would mark those remaining items as resolved when they are not.
  Remediation: Change Closes #6512 to Ref #6512 or Part of #6512 in the PR body so the issue remains open for the remaining workflow-file fixes.

Medium

• [protected-path] .github/scripts/check-agents-gate-pin.sh, .github/scripts/check-agents-gate-pin-test.sh — PR adds files under the .github/ protected path. The PR links to issue release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512 and the maintainer explicitly authorized the drift-check script and tests in a rescoping comment. Human approval is always required for protected-path changes, regardless of context.

Low

• [design-direction] renovate.json — The custom regex manager tracking fullsend-ai/agents main via git-refs may become dead config if the long-term fix in issue release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512 eliminates the SHA pin by moving to uses: ...@main. The maintainer explicitly requested this Renovate work for the interim period while the pin exists; a future PR removing the pin should also remove the tracker.

---

Next steps:

• /fs-fix — agent addresses review findings automatically
• /fs-fix <your instruction> — agent fixes with your specific guidance
• Push commits directly — review re-runs automatically on push
• /fs-fix-stop — disable automatic fix runs for this PR

Previous run (6)

Review

Findings

Critical

• [unauthorized-change] renovate.json — The PR uses Closes #6512 but issue release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512 explicitly places this PR out of scope: "PR ci(#6512): fix validate-agents caller permissions and tag the resolved SHA #6513 implements this and should not be merged as the fix." The issue identifies four items to fix (caller permissions, checkout ref, re-resolution of main, rollup job) — none addressed by this PR. Merging would auto-close release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512 via GitHub's keyword linkage, silently marking all four actual bugs as resolved when they are not.
  Remediation: Change Closes #6512 to Related to #6512 or Ref #6512 in the PR body.

Medium

• [design-direction] renovate.json — The custom regex manager tracking fullsend-ai/agents main via git-refs contradicts the architectural direction in issue release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512. The issue's proposed fix eliminates the SHA pin entirely by moving to uses: ...@main, making the Renovate tracker unnecessary. If both the issue fix and this PR land, the Renovate manager becomes dead config.

• [protected-path] .github/scripts/check-agents-gate-pin.sh, .github/scripts/check-agents-gate-pin-test.sh — PR modifies files under the .github/ protected path. The PR links to issue release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512 and provides clear rationale for the changes. Human approval is always required for protected-path changes, regardless of context.

Low

• [edge-case] .github/scripts/check-agents-gate-pin.sh:27 — If release.yml contains more than one line matching the grep pattern, PINNED_SHA will contain newline-separated SHAs. The equality check against AGENTS_MAIN_SHA would always fail, producing a misleading "stale pin" error rather than surfacing the ambiguity.

• [gha-workflow-command-injection] .github/scripts/check-agents-gate-pin.sh:20 — RELEASE_YML environment variable is interpolated into GHA ::error:: workflow commands without sanitization for :: sequences or encoded newlines. The variable defaults to a safe static path, and the other interpolated variables are regex-constrained or API-sourced.

• [scope-creep] .github/scripts/check-agents-gate-pin.sh — The drift-check script and tests (272 new lines) are useful standalone tooling, but they are not authorized by issue release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512.

• [scope-exceeded] internal/scaffold/vendormanifest.go — The vendormanifest.go change registers the new scripts in vendoredDefaultsInfraPaths. This is a mechanical consequence of adding the scripts.

• [pattern-inconsistency] .github/scripts/check-agents-gate-pin.sh — GitHub Actions annotations (::error::) are redirected to stderr (>&2) on lines 19, 34, 39, 47, 52, and 69, while the established convention writes annotations to stdout. GitHub Actions only parses annotations from stdout, so >&2 suppresses them from the Actions UI. The ::notice:: on line 62 correctly uses stdout, creating internal inconsistency.

• [pattern-inconsistency] .github/scripts/check-agents-gate-pin-test.sh — The test harness does not set GH_TOKEN in the environment when invoking the script under test. The established pattern in check-fix-eligibility-test.sh passes GH_TOKEN="fake".

---

Labels: PR adds CI scripts under .github/scripts/ and modifies Renovate CI configuration

---

Next steps:

• /fs-fix — agent addresses review findings automatically
• /fs-fix <your instruction> — agent fixes with your specific guidance
• Push commits directly — review re-runs automatically on push
• /fs-fix-stop — disable automatic fix runs for this PR

Previous run (7)

Review

Findings

Medium

• [consumer-completeness] renovate.json:21 — The narrowed matchPackageNames pattern (/^fullsend-ai\/fullsend($|\/)/) no longer excludes fullsend-ai/agents from the native github-actions manager. Since managerFilePatterns extends (not replaces) default file patterns, the native manager still scans .github/workflows/release.yml. Both the native github-actions manager and the new custom regex manager will independently track fullsend-ai/agents, potentially producing duplicate or conflicting Renovate PRs for the same SHA update.
  Remediation: Add a packageRules entry disabling the github-actions manager specifically for fullsend-ai/agents: {"matchManagers": ["github-actions"], "matchPackageNames": ["fullsend-ai/agents"], "enabled": false}.

• [protected-path] .github/scripts/check-agents-gate-pin.sh, .github/scripts/check-agents-gate-pin-test.sh — PR modifies files under the .github/ protected path. The PR links to issue release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512 and provides clear rationale for the changes. Human approval is always required for protected-path changes, regardless of context.

Low

• [error-handling-idiom] .github/scripts/check-agents-gate-pin.sh:27 — The grep -oE ... || true pattern suppresses all grep exit codes, including exit code 2 (file-read errors), not just exit code 1 (no match). In practice the preceding file-existence check on line 18 mitigates most scenarios, making this a minor robustness gap rather than a functional defect.

• [scope-description-accuracy] renovate.json:19 — The description field still reads "Ignore fullsend self-references (own reusable workflows and actions)" after narrowing the regex from all fullsend-ai/* to only fullsend-ai/fullsend. While still technically accurate, the text could be more precise to reflect the new scope.

---

Next steps:

• /fs-fix — agent addresses review findings automatically
• /fs-fix <your instruction> — agent fixes with your specific guidance
• Push commits directly — review re-runs automatically on push
• /fs-fix-stop — disable automatic fix runs for this PR

Previous run (8)

Review

Findings

Medium

• [consumer completeness] renovate.json:21 — The narrowed matchPackageNames pattern (/^fullsend-ai\/fullsend($|\/)/) no longer excludes fullsend-ai/agents from the native github-actions manager. Since managerFilePatterns extends (not replaces) default file patterns, the native manager still scans .github/workflows/release.yml. Both the native github-actions manager and the new custom regex manager will independently track fullsend-ai/agents, potentially producing duplicate or conflicting Renovate PRs for the same SHA update.
  Remediation: Add a packageRules entry disabling the github-actions manager specifically for fullsend-ai/agents: {"matchManagers": ["github-actions"], "matchPackageNames": ["fullsend-ai/agents"], "enabled": false}.

• [protected-path] .github/scripts/check-agents-gate-pin.sh, .github/scripts/check-agents-gate-pin-test.sh — PR modifies files under the .github/ protected path. The PR links to issue release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512 and provides clear rationale for the changes. Human approval is always required for protected-path changes, regardless of context.

Low

• [error handling] .github/scripts/check-agents-gate-pin.sh:27 — The grep -oE ... || true pattern suppresses all grep exit codes, including exit code 2 (file-read errors), not just exit code 1 (no match). In practice the preceding file-existence check on line 18 mitigates most scenarios, making this a minor robustness gap rather than a functional defect.

• [scope-description-accuracy] renovate.json:19 — The description field still reads "Ignore fullsend self-references (own reusable workflows and actions)" after narrowing the regex from all fullsend-ai/* to only fullsend-ai/fullsend. While still technically accurate, the text could be more precise to reflect the new scope.

[ralphbean]

/fs-fix Fix the failing tests in CI (but, I think the functional test failure is just a flake, so that one can be ignored).

[fullsend-ai-coder]

🤖 Finished Fix · ✅ Success · Started 1:08 PM UTC · Completed 1:18 PM UTC

Commit: 6582af4 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $1.73

[fullsend-ai-coder]

🔧 Fix agent — iteration 1 (human-triggered)

The new .github/scripts/check-agents-gate-pin{,-test}.sh files were not registered in vendoredDefaultsInfraPaths, causing two scaffold tests to fail. Added both paths in alphabetical order. The functional-tests failure was identified by the human as a flake and skipped per instruction.

Fixed (1):

1. failing Go tests in CI: TestVendoredDefaultsInfraPathsMatchPredicate and TestEnumerateVendoredPathsMatchesCollectInCheckout (internal/scaffold/vendormanifest.go): Added check-agents-gate-pin.sh and check-agents-gate-pin-test.sh to vendoredDefaultsInfraPaths so the hardcoded list matches the files discovered on disk by walkVendoredUpstreamFromRoot

Disagreed (1):

1. functional-tests CI check failure: Human instruction explicitly stated the functional test failure is a flake and should be ignored

Tests: passed

Next steps:

• /fs-review — request a re-review of the changes
• /fs-fix <your instruction> — run another fix pass with specific guidance
• Push commits directly — review re-runs automatically on push
  _{Updated by fullsend fix agent}

[fullsend-ai-review]

🤖 Review · ❌ Terminated · Started 1:19 PM UTC · Ended 1:34 PM UTC

Commit: b2f6d32 · View workflow run →

[codecov]

Codecov Report

❌ Patch coverage is 83.92857% with 9 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/cli/vendor.go	75.00%	4 Missing and 3 partials ⚠️
internal/cli/admin.go	0.00%	1 Missing ⚠️
internal/cli/github.go	0.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 1:19 PM UTC · Completed 1:34 PM UTC

Commit: b2f6d32 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $5.11

[fullsend-ai-coder]

🤖 Finished Fix · ✅ Success · Started 1:36 PM UTC · Completed 1:42 PM UTC

Commit: b2f6d32 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $1.84

[fullsend-ai-coder]

🔧 Fix agent — iteration 2 (bot-triggered)

Addressed 3 of 4 review findings with code fixes. Disagreed with 1 (protected-path is an informational/process finding, not an actionable code change). Added a packageRules entry to prevent duplicate Renovate tracking, improved grep error handling to distinguish no-match from file-read errors, and updated the ignore-rule description for accuracy.

Fixed (3):

1. consumer-completeness: duplicate Renovate tracking of fullsend-ai/agents (renovate.json): Added a packageRules entry disabling the native github-actions manager for fullsend-ai/agents, so only the custom regex manager (git-refs datasource) tracks the pinned SHA. This prevents duplicate or conflicting Renovate PRs.
2. error-handling-idiom: grep || true suppresses all exit codes including file-read errors (.github/scripts/check-agents-gate-pin.sh): Replaced broad || true with || grep_rc=$? and an explicit check: exit code 1 (no match) falls through to the existing empty-check, while exit code >= 2 (file-read or internal error) surfaces the failure and exits non-zero.
3. scope-description-accuracy: ignore-rule description no longer matches narrowed regex (renovate.json): Updated the description from 'Ignore fullsend self-references' to 'Ignore fullsend-ai/fullsend self-references' to accurately reflect that the pattern now targets only the fullsend repo, not all fullsend-ai/* repos.

Disagreed (1):

1. protected-path: .github/scripts/ files require human approval: This is an informational finding, not an actionable code change. The PR already links to issue release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512 with clear rationale. Human approval for protected-path changes is a process gate handled outside the fix agent.

Tests: passed

Next steps:

• /fs-review — request a re-review of the changes
• /fs-fix <your instruction> — run another fix pass with specific guidance
• Push commits directly — review re-runs automatically on push
  _{Updated by fullsend fix agent}

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 1:44 PM UTC · Completed 2:04 PM UTC

Commit: a6b5598 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $5.59

[fullsend-ai-coder]

🤖 Finished Fix · ✅ Success · Started 2:06 PM UTC · Completed 2:15 PM UTC

Commit: a6b5598 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $2.20

[fullsend-ai-coder]

🔧 Fix agent — iteration 3 (bot-triggered)

Addressed 5 of 9 review findings with code fixes: GHA annotation stderr→stdout routing, RELEASE_YML sanitization in annotations, multi-pin ambiguity detection with test, and GH_TOKEN test convention. The critical unauthorized-change finding (Closes→Ref in PR body) was verified correct but requires manual PR body edit — sandbox network policy blocks gh pr edit. Disagreed with 4 findings (design-direction, protected-path acknowledgment, scope-creep, scope-exceeded) with reasoned justifications.

Fixed (5):

1. unauthorized-change: Closes release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512 auto-closes issue prematurely: Verified finding is correct — PR body uses 'Closes release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512' which would auto-close the issue via GitHub keyword linkage. Sandbox network policy blocks gh pr edit, so the PR body change (Closes → Ref) requires manual update.
2. edge-case: multiple grep matches produce misleading stale-pin error (.github/scripts/check-agents-gate-pin.sh): Added ambiguity detection: if grep returns multiple SHA matches, the script now exits with an explicit 'Ambiguous: found N pins' error instead of silently failing the equality check. Added corresponding test case.
3. gha-workflow-command-injection: RELEASE_YML interpolated unsanitized in annotations (.github/scripts/check-agents-gate-pin.sh): Sanitized RELEASE_YML in all GHA annotation strings using ${RELEASE_YML//::/} to strip :: sequences that could inject false workflow commands.
4. pattern-inconsistency: GHA annotations redirected to stderr (.github/scripts/check-agents-gate-pin.sh): Removed >&2 from all ::error:: annotation lines. GitHub Actions only parses annotations from stdout; the ::notice:: on the success path already used stdout correctly.
5. pattern-inconsistency: test harness missing GH_TOKEN (.github/scripts/check-agents-gate-pin-test.sh): Added GH_TOKEN="fake" to both run_test() and run_test_missing_file() invocations, matching the established convention in check-fix-eligibility-test.sh.

Disagreed (4):

1. design-direction: Renovate tracker contradicts issue release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512 direction: The Renovate tracker serves its purpose while the SHA pin exists. If a future PR eliminates the pin (per issue release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512's proposed fix), that PR should also remove the Renovate manager entry. Adding the tracker now and removing it later is the correct sequencing — it prevents the pin from silently drifting in the interim.
2. protected-path: .github/ files require human approval: Acknowledged — this is informational. The review agent correctly notes that human approval is always required for protected-path changes regardless of context. No code change needed.
3. scope-creep: drift-check script not authorized by issue release: validate-agents startup failure (caller permissions) + agents gate validates a different tree than tag-agents tags #6512: The PR summary explicitly lists the drift-check script as an in-scope deliverable. The issue mentions 'a standalone drift-check script for manual or future CI use' as part of this PR's purpose.
4. scope-exceeded: vendormanifest.go registers new scripts: The reviewer acknowledges this is a 'mechanical consequence of adding the scripts.' Registering new infra scripts in vendoredDefaultsInfraPaths is a required step, not scope creep.

Tests: passed

Next steps:

• /fs-review — request a re-review of the changes
• /fs-fix <your instruction> — run another fix pass with specific guidance
• Push commits directly — review re-runs automatically on push
  _{Updated by fullsend fix agent}

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 9:47 PM UTC · Ended 9:49 PM UTC

Commit: 5a1b639 · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 9:50 PM UTC · Completed 10:12 PM UTC

Commit: 72881a7 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $6.87

[fullsend-ai-review]

Note: The following review comments could not be posted on the diff (GitHub returned 422) and are included here instead:

• internal/cli/vendor_test.go (file-level): Line 115 · [low] constructor-convention

Pre-existing tests TestAcquireAndVendor_ExplicitPath and TestAcquireAndVendor_CheckoutBuild use &forge.FakeClient{} while the rest of the package uses forge.NewFakeClient(). Not a regression but a missed normalization opportunity.

[fullsend-ai-review]

See the review comment for full details.

[waynesun09]

Maintainer takeover reviewed end-to-end: release permissions fix validated against the v0.37.0 failure evidence, resolve-once tagging, ship-set allowlists with prune verified on all vendor commit paths (Grok review round 2 critical fixed), all threads addressed.

[fullsend-ai-retro]

🤖 Finished Retro · ✅ Success · Started 11:09 PM UTC · Completed 11:22 PM UTC

Commit: 72881a7 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $4.44

[fullsend-ai-retro]

Retro: PR #6513 — ci(#6512): fix validate-agents caller permissions and tag the resolved SHA

Timeline

Issue #6512 reported release pipeline defects requiring .github/workflows/release.yml changes. The code agent was dispatched but hit the workflows permission boundary on its first attempt (run 32614319356). The maintainer rescoped to pushable work (Renovate config, drift-check scripts), and the code agent succeeded on the second attempt (run 32614776947), creating PR #6513. The review agent ran 9 times ($56) and the fix agent ran 4 iterations ($7). The maintainer then took over the PR — reverting the agent's Renovate approach, implementing the actual release fix, and adding vendored ship-set improvements. The human reviewer triaged all 14 agent-generated review threads and approved after end-to-end verification. PR merged ~44 hours after issue creation.

What went well

• Triage agent correctly identified needs-human after the code agent completed pushable work but the release blocker still required workflow file changes the agent cannot push.
• Fix agent performed well across 4 iterations: fixed real bugs (duplicate Renovate tracking, grep error handling, GHA annotation routing), correctly escalated a sandbox limitation (cannot edit PR body via API) in iteration 4, and properly disagreed with informational findings like [protected-path].
• Review agent found one genuinely valuable bug — [consumer-completeness] identifying a duplicate Renovate tracking risk — that the fix agent resolved in iteration 2.
• Human review was triage-focused: waynesun09 responded to all 14 agent-generated threads but opened zero independent finding threads, suggesting the review agent's coverage was comprehensive for this class of CI/workflow change.

Cost concerns and evidence for existing issues

The review agent's 9 runs at ~$56 were the dominant cost. Root causes map to existing issues:

• Repeated unfixable findings: [protected-path] raised identically in all 9 runs — evidence for #1392 and #2794.
• Inline comment duplication: Same comments re-posted across runs on the same or similar code — evidence for #5007 and #1285.
• False Critical severity: [unauthorized-change] was rated Critical in run 3 based on misreading the issue's evolution (the issue had been rescoped via comments, but the agent read the original body's scope statement). Downgraded to High in run 4. Evidence for severity calibration issues (agents#741).
• Review runs on rapid human force-pushes: 5 of the 9 review runs (~$34) were triggered by the maintainer's rapid succession of force-pushes during takeover. No existing issue covers this — see proposal below.

Autonomy observations

The human reviewer opened zero independent finding threads — all 14 were agent-initiated. Human value was in judgment and disposition: dismissing 4 threads as moot (agent reviewed reverted code), making 2 substantive architectural accept/defer decisions, and citing specific commit SHAs for every resolution. The review agent's discovery coverage is strong for CI/workflow PRs, but human oversight remains essential for severity calibration and architectural judgment calls.

Proposals filed

• Debounce review dispatch during rapid human pushes to PR branches (in fullsend-ai/fullsend)