fix(#7231): remint GitHub App token before post-script - #7234
Conversation
GitHub App installation tokens expire after 60 minutes, matching the code agent's budget. The post-script runs after that budget is spent and after sandbox teardown, so a full-budget run authenticated with an expired token and could not push. Re-mint the agent token in the post-script defer after teardown and before postScriptEnv, overlaying the fresh values onto RunnerEnv so the last-wins merge does not restore the token expanded at start of run. A remint failure is non-fatal: the post-script still runs with the existing token. Closes #7231
|
🤖 Finished Review · ✅ Success · Started 12:40 PM UTC · Completed 1:01 PM UTC Commit: Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $6.73 |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
Risk Assessment: moderate (2/5) DetailsRe-review anchoring: Tier 1 signals unchanged from the prior assessment at d65b055 (bot author, no protected/security/CI/dependency paths, 50% test ratio, large blast radius driven by size not qualitative risk); Tier 2 again shows run.go/run_test.go as high-churn hub files with the mintclient files low-churn; Tier 3 remains a tight, well-matched fix for the linked production incident with no new risk-relevant issue labels. This diff is a small additive increment (new exported timeout const with doc comment, an Unwrap method, and tests) on top of the already-assessed state rather than new risk surface, so the composite stays at the prior anchor score of 2 (moderate). Previous runRisk Assessment: moderate (2/5) DetailsTier 1 stays low overall (bot author, no protected/security/CI/dependency paths, 50% test ratio) though LINES_CHANGED crossed into the 300-799 band; Tier 2 confirms run.go/run_test.go are high-churn, high-author-contention hub files; Tier 3 shows a tight, well-matched and now further-hardened fix for the linked production incident. The composite matches the prior anchor score of 2 (moderate) since the added lines are additive hardening (race-condition and context-cancellation fixes plus more tests) addressing prior review feedback rather than new risk surface. Previous run (2)Risk Assessment: moderate (2/5) DetailsTier 1 metadata is uniformly low risk (tiny, well-tested, non-protected, bot-authored fix with no CI/dependency changes), and the linked issue shows a tight scope match with the fix directly addressing the reported production incident, but the touched file (internal/cli/run.go) is an extremely high-churn, high-author-contention hub with dense change coupling to many untouched files and no feature flag guarding the new remint behavior, which pulls the composite up from a Tier-1-only baseline of ~1 to a moderate 2. |
|
Looks good to me Previous runLooks good to me Previous run (2)ReviewFindingsLow
Info
Next steps:
|
|
🤖 Finished Fix · ✅ Success · Started 1:03 PM UTC · Completed 1:27 PM UTC Commit: Runtime: claude · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $2.40 |
…handling Fixes three findings from the automated review on PR #7234: - Stop the run-scoped OpenAI credential refresher goroutines before the post-script defer's token remint. Their stop-defers are registered earlier in runAgent than the post-script defer, so under LIFO ordering they would not fire until after the remint's os.Setenv calls, racing against the refresher's os.Getenv reads. The comment claiming all env-reading goroutines were already torn down was accurate for OIDC and sandbox streaming (registered after the post-script defer) but not for this refresher -- now stopped explicitly, and the comment corrected. - Mint the post-script token on a context derived from context.WithTimeout(context.WithoutCancel(ctx), ...), mirroring the completion-notification defer's existing pattern, so a parent-ctx cancellation near the run's own budget can't abort the remint before it gets a chance to complete. - Add tests for the GitLab/empty-mintURL early return (asserts zero mint calls and an untouched RunnerEnv) and for a RunnerEnv missing the token keys entirely (asserts postScriptEnv still resolves the reminted process-env value via last-wins), plus a direct test of syncRunnerEnvTokens' nil-harness/nil-RunnerEnv guards. Addresses #7234
🔧 Fix agent — iteration 1 (bot-triggered)Fixed the two Low-severity correctness findings (the OpenAI-refresher/remint os.Setenv race, and the cancellable-context remint that could be aborted near the run's timeout) and the Low-severity missing-test finding (GitLab/empty-mintURL skip path, RunnerEnv-missing-keys path). Added a test for the Info-level syncRunnerEnvTokens nil-guards gap. Disagreed with testing the Info-level remintAgentTokenForPostScript cleanup==nil branch, which is unreachable dead code given mintAgentToken's actual (nil cleanup, nil err) contract. The remaining Info-level items (scope-authorization-verified, design-coherence, forge-abstraction, commit-classification) were pure affirmations with no remediation requested, so no action was needed for them. Fixed (4):
Disagreed (1):
Tests: passed Decision points
Next steps:
|
|
🤖 Finished Review · ✅ Success · Started 1:29 PM UTC · Completed 1:41 PM UTC Commit: Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $3.51 |
Superseded by updated review
|
/fs-fix Do not sign off. No Two findings to fix. Keep the change inside [MEDIUM] — The 30 s remint bound is shorter than the mint client's own retry schedule, so a slow mint service silently reintroduces #7231File: [MEDIUM] — The defer ordering and cancelled-parent behaviour are shipped without a test that exercises themFile: |
|
🤖 Finished Fix · ✅ Success · Started 3:43 PM UTC · Completed 4:00 PM UTC Commit: Runtime: claude · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $2.57 |
…ellation survival Addresses review feedback on PR #7234: - remintForPostScriptTimeout now uses the new mintclient.MaxMintDuration (120s, documented against fetchOIDCJWT/callMint's actual retry+backoff schedule) instead of an arbitrary 30s, so the bound no longer routinely cuts off retries mid-backoff and falls through to the expired token this remint exists to replace. - retryableError now implements Unwrap so errors.Is(err, context.DeadlineExceeded) works even when the deadline lands mid-request rather than between retries; remintAgentTokenForPostScript uses this to emit a distinct "timed out" warning instead of the generic mint-failure message. - The context.WithTimeout(context.WithoutCancel(ctx), ...) wrapping moved from the runAgent call site into remintAgentTokenForPostScript itself, so a test can pass an already-cancelled parent ctx directly and observe the remint still run. - Added tests: cancelled-parent-ctx survival, distinct deadline-exceeded warning, restoration order after remintCleanup() vs. the first mint's cleanup(), and a mintclient-level test that shortens retryBaseDelay to prove MintToken survives several transient 5xx responses within MaxMintDuration. Not implemented: a runAgent-level test driving a real post-script process through the fake-openshell stub to observe the second minted token land in the child's env. The existing stubs (testdata/openshell, testdata/providers-stub) fail before reaching sandbox streaming and extraction, and building one that carries a full run to the post-script defer is a disproportionate addition for this fix; disclosing per the review's own suggestion. Addresses #7234
🔧 Fix agent — iteration 2 (human-triggered)Fixed both MEDIUM findings: derived remintForPostScriptTimeout from a new, documented mintclient.MaxMintDuration (120s) instead of an arbitrary 30s, made retryableError unwrappable so a timed-out remint gets a distinct warning, and moved the ctx-wrapping into remintAgentTokenForPostScript so cancellation-survival is directly testable. Added four new tests covering the timeout derivation, distinct warning, cancelled-parent-ctx survival, and defer-restoration order, plus extended the mintclient test suite with a transient-failure survival test. Disclosed one sub-item (a full runAgent-level integration test through the post-script) as out of proportion for this fix, per the review's own escape hatch. Fixed (2):
Tests: passed Decision points
Next steps:
|
|
🤖 Finished Review · ✅ Success · Started 4:02 PM UTC · Completed 4:18 PM UTC Commit: Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $6.11 |
Superseded by updated review
waynesun09
left a comment
There was a problem hiding this comment.
Approving at 8881591 after three review rounds.
The runtime fix is correct and I verified its premises: the remint runs after sandbox teardown on a cancellation-immune, bounded context; syncRunnerEnvTokens defeats postScriptEnv's last-wins merge, which is what makes the fresh token actually reach the child; cleanup ordering under LIFO restores the env correctly; the cancel func is deferred on every return path; the second token is masked; GitLab is skipped; and fix/review/retro/triage take the same path with consistent role→token mapping. Nothing regresses relative to main, and this closes the failure mode that discarded a complete run on #7218.
Three non-blocking follow-ups, each a one-liner or doc-only, worth a small follow-up PR:
TestRemintAgentTokenForPostScript_SurvivesCancelledParentCtxis currently vacuous. Its fake mint discardsctx, andmintAgentTokenWithRetrycalls the mint before ever checkingctx.Err(), so the test still passes withcontext.WithoutCancelremoved. Fix: have the fake returnctx.Err()when non-nil before incrementingcalls.- The "timed out" warning misfires on a final-attempt
Client.Timeout.errors.Is(err, context.DeadlineExceeded)now sees through the newretryableError.Unwrap, and net/http's timeout error deliberately matchesDeadlineExceeded, so a run that exhausts the client's full schedule is reported as bound truncation. Fix: discriminate onremintCtx.Err() != nilinstead; the existing test stays green. MaxMintDurationdoc names the wrong caller and states a premise as fact. The bound enclosesmintAgentTokenWithRetry(up to 4MintTokencalls, +14 s backoff), not a singleMintToken, and "outages fail fast" is an assumption; under a hanging service the bound truncates at mint attempt 4. Either state the assumption explicitly and correct the caller name, or extract the attempt counts as consts and assert the invariant in a test.
Also noted, minor: the transient-failure test cannot fail on the constant it names (40 ms of exercised backoff vs a 120 s bound), and syncRunnerEnvTokens refreshes only same-named keys, which is fine for every current harness but worth a sentence in its doc.
|
🤖 Finished Retro · ✅ Success · Started 6:18 PM UTC · Completed 6:28 PM UTC Commit: Runtime: claude · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $2.11 |
|
PR #7234 (fixing #7231: the GitHub App installation token expiring before the post-script ran on full-budget code-agent runs) went through triage → code → 3 review/fix rounds → merge, all within ~6 hours on 2026-09-11, at ~$21 total agent spend. The triage agent correctly diagnosed the root cause but misjudged scope, flagging The review agent's round-1 pass found 3 legitimate Low-severity issues (a goroutine race with a newly-added OpenAI credential refresher, a cancellable-context bug, missing tests) that the fix agent addressed correctly — genuine value-add, not noise. Rounds 2 and 3 both returned a bare 'Looks good to me'; this is the harness's mandated zero-findings output format, not evidence of a skipped review — the agents-repo review skill (fullsend-ai/agents@791d281c, skills/pr-review + skills/code-review) keeps the 'correctness' dimension at full scope on every re-review round by design. Despite that, the human reviewer found a MEDIUM issue after round 2 (an unjustified 30s remint timeout, shorter than the mint client's own retry schedule) and, after round 3, three more non-blocking issues: a vacuous cancellation test (matches open issue fullsend-ai/agents#681 on verifying test-assertion effectiveness — new corroborating instance, no new issue filed), a doc comment naming the wrong function, and — most notably — a new That last issue is the third dated occurrence of the identical Timeout()/DeadlineExceeded/Unwrap confusion in this repo (after PR #6217 and its follow-ups #6424 and #6425, both closed within the last 3 weeks) — documentation and single-instance hardening haven't stopped it recurring in new code, and it slipped past 3 rounds of automated review whose correctness dimension runs at full scope every round. I'm filing one proposal to consolidate the ~5 independent hand-rolled implementations of this check into a single shared, regression-tested helper, since that closes the gap regardless of whether the review agent's checklist happens to catch the next instance. I did not file additional narrow 'correctness sub-agent should check X' issues for the timeout-derivation or vacuous-test findings: fullsend-ai/agents already carries a very large (30+) backlog of similarly narrow, seemingly-unimplemented correctness-checklist issues, and adding to that pile seemed lower value than the consolidation fix and the evidence notes above. Proposals filed
|
Summary
Re-mint the GitHub App installation token after the sandbox is torn down and before the post-script runs. Installation tokens live 60 minutes, matching the code agent's budget, so a full-budget run previously handed the post-script an expired token and could not push or post a failure comment.
Related Issue
#7231
Changes
mintAgentTokenagain from the post-script defer inrunAgentwhen the forge is not GitLab and a mint URL is set.GH_TOKEN/PUSH_TOKEN(and other role token vars) ontoh.RunnerEnvsopostScriptEnv's last-wins merge does not restore the token snapshotted at the start of the run.Testing
gofmtandgo vet ./...passgo test -racefor remint and existing mint tests ininternal/climake lint(pre-commit could not fetch remote hook repos in this sandbox; equivalent local hooks were run)Checklist
!for breaking changes)Closes #7231
Post-script verification
agent/7231-remint-post-script-token)25d08527c7c4dc5cbfea640923be43510d420fa2..HEAD)