Skip to content

fix(supervisor): detect gh auth failures to prevent clean_exit_no_signal retry waste (t198) #924

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
marcusquinn wants to merge 1 commit into from
Closed

fix(supervisor): detect gh auth failures to prevent clean_exit_no_signal retry waste (t198) #924

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .agents/scripts/supervisor-helper.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4766,6 +4766,19 @@ extract_log_metadata() {
fi
echo "task_obsolete=$task_obsolete"

# GitHub auth failure detection (t198): workers that complete their work but
# can't push/create a PR because gh auth is expired. This is the #1 cause of
# clean_exit_no_signal (79% of cases in production logs). Without this check,
# the supervisor retries them 3x at Opus cost, each failing identically.
# Search the FULL log (not just tail) because auth failures appear in tool
# output mid-log, not at the end. This is safe because gh auth patterns are
# specific enough to avoid false positives from documentation content.
local gh_auth_failure="false"
if grep -qiE 'gh auth (status|login).*fail|authentication token.*expired|not logged in|try authenticating|gh: To use .* in a non-interactive context|could not authenticate|failed to authenticate' "$log_file" 2>/dev/null; then
gh_auth_failure="true"
fi
Comment on lines +4777 to +4779
Copy link

@gemini-code-assist gemini-code-assist bot Feb 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This use of 2>/dev/null to suppress grep's error output violates the repository's style guide (line 50), which states that 2>/dev/null is only acceptable when redirecting to log files. A safer and more idiomatic way to handle cases where the log file might not exist is to check for the file's existence before calling grep.

Suggested change
if grep -qiE 'gh auth (status|login).*fail|authentication token.*expired|not logged in|try authenticating|gh: To use .* in a non-interactive context|could not authenticate|failed to authenticate' "$log_file" 2>/dev/null; then
gh_auth_failure="true"
fi
if [[ -f "$log_file" ]] && grep -qiE 'gh auth (status|login).*fail|authentication token.*expired|not logged in|try authenticating|gh: To use .* in a non-interactive context|could not authenticate|failed to authenticate' "$log_file"; then
gh_auth_failure="true"
fi
References
  1. The style guide at line 50 bans blanket suppression of stderr with 2>/dev/null, allowing it only when redirecting to log files. The current code uses it to hide potential 'file not found' errors from grep. (link)

echo "gh_auth_failure=$gh_auth_failure"
Comment on lines +4769 to +4780
Copy link
Contributor

@coderabbitai coderabbitai bot Feb 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Tighten auth-failure regex to avoid false positives in full-log scan.

Generic phrases like “not logged in” can appear in docs or code samples within logs, which would incorrectly block a task. Scope matches to gh/git auth error lines (e.g., gh auth, gh:, fatal: Authentication failed, remote: Permission denied) before setting gh_auth_failure=true.

Suggested refinement 
-    if grep -qiE 'gh auth (status|login).*fail|authentication token.*expired|not logged in|try authenticating|gh: To use .* in a non-interactive context|could not authenticate|failed to authenticate' "$log_file" 2>/dev/null; then
+    if grep -qiE '(gh auth (status|login).*(fail|not logged in|authenticate)|gh:.*(not logged in|authenticate)|GitHub CLI.*(not logged in|authenticate)|fatal:.*authentication failed|remote:.*(authentication|permission) denied)' "$log_file" 2>/dev/null; then
         gh_auth_failure="true"
     fi
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# GitHub auth failure detection (t198): workers that complete their work but
# can't push/create a PR because gh auth is expired. This is the #1 cause of
# clean_exit_no_signal (79% of cases in production logs). Without this check,
# the supervisor retries them 3x at Opus cost, each failing identically.
# Search the FULL log (not just tail) because auth failures appear in tool
# output mid-log, not at the end. This is safe because gh auth patterns are
# specific enough to avoid false positives from documentation content.
local gh_auth_failure="false"
if grep -qiE 'gh auth (status|login).*fail|authentication token.*expired|not logged in|try authenticating|gh: To use .* in a non-interactive context|could not authenticate|failed to authenticate' "$log_file" 2>/dev/null; then
gh_auth_failure="true"
fi
echo "gh_auth_failure=$gh_auth_failure"
# GitHub auth failure detection (t198): workers that complete their work but
# can't push/create a PR because gh auth is expired. This is the `#1` cause of
# clean_exit_no_signal (79% of cases in production logs). Without this check,
# the supervisor retries them 3x at Opus cost, each failing identically.
# Search the FULL log (not just tail) because auth failures appear in tool
# output mid-log, not at the end. This is safe because gh auth patterns are
# specific enough to avoid false positives from documentation content.
local gh_auth_failure="false"
if grep -qiE '(gh auth (status|login).*(fail|not logged in|authenticate)|gh:.*(not logged in|authenticate)|GitHub CLI.*(not logged in|authenticate)|fatal:.*authentication failed|remote:.*(authentication|permission) denied)' "$log_file" 2>/dev/null; then
gh_auth_failure="true"
fi
echo "gh_auth_failure=$gh_auth_failure"
🤖 Prompt for AI Agents 
In @.agents/scripts/supervisor-helper.sh around lines 4769 - 4780, The current
full-log grep that sets gh_auth_failure=true is too broad and matches generic
phrases; update the grep in the block that defines gh_auth_failure to restrict
matches to explicit auth failure lines only (references: variable
gh_auth_failure, the grep invocation using "$log_file"): match patterns like
'^.*gh auth (status|login).*fail', '(^|[:space])gh:', 'fatal: Authentication
failed', 'remote: Permission denied', and 'authentication token.*expired'
(case-insensitive), and remove or de-prioritize generic phrases such as "not
logged in" or "failed to authenticate" unless they appear in those auth-specific
contexts; keep the 2>/dev/null and echo "gh_auth_failure=$gh_auth_failure"
behavior.


# Task tool parallelism tracking (t217): detect whether the worker used the
# Task tool (mcp_task) to spawn sub-agents for parallel work. This is a
# heuristic quality signal — workers that parallelise independent subtasks
Expand Down Expand Up @@ -5139,6 +5152,20 @@ evaluate_worker() {
fi
fi

# GitHub auth failure detection (t198): workers that complete their work but
# can't push/create a PR because gh auth is expired or invalid. This is the
# #1 cause of clean_exit_no_signal (79% of cases). Retrying is pointless —
# the auth issue persists across retries. Block immediately so the human can
# fix credentials, then reset the task.
if [[ "$meta_exit_code" == "0" && "$meta_signal" == "none" ]]; then
local meta_gh_auth_failure
meta_gh_auth_failure=$(_meta_get "gh_auth_failure" "false")
if [[ "$meta_gh_auth_failure" == "true" ]]; then
echo "blocked:gh_auth_expired"
return 0
fi
fi

# Clean exit with no completion signal and no PR (checked DB + gh API above)
# = likely incomplete. The agent finished cleanly but didn't emit a signal
# and no PR was found. Retry (agent may have run out of context or hit a
Expand Down
26 changes: 26 additions & 0 deletions tests/test-dispatch-worktree-evaluate.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -834,6 +834,32 @@ else
fail "Backend error in retry log should be backend_quota_error" "Got: $eval_result"
fi

# ============================================================
# SECTION: GitHub Auth Failure Detection (t198)
# ============================================================
section "GitHub Auth Failure Detection (t198)"

# Integration test: worker completes work but gh auth is expired
sup add integ-t198b --repo "$TEST_REPO" --description "GH auth failure test" --no-issue >/dev/null
sup transition integ-t198b dispatched >/dev/null
sup transition integ-t198b running >/dev/null

create_log "integ-t198b" 'WORKER_STARTED task_id=integ-t198b pid=12345 timestamp=2026-02-09T03:00:00Z
{"type":"step_start","timestamp":1770606000000,"part":{"type":"step-start"}}
{"type":"text","timestamp":1770606100000,"part":{"type":"text","text":"All implementation complete. Files created and committed."}}
{"type":"tool_use","timestamp":1770606200000,"part":{"type":"tool","tool":"bash","state":{"status":"completed","input":{"command":"gh auth status"},"output":"You are not logged in to any GitHub hosts. Run gh auth login to authenticate.","metadata":{"exit":1}}}}
{"type":"text","timestamp":1770606300000,"part":{"type":"text","text":"GitHub authentication has expired. Cannot create PR."}}
{"type":"step_finish","timestamp":1770606400000,"part":{"type":"step-finish","reason":"stop"}}
EXIT:0' >/dev/null

sup transition integ-t198b evaluating >/dev/null
eval_result=$(sup evaluate integ-t198b --no-ai 2>&1 | grep "^Verdict:" || echo "")
if echo "$eval_result" | grep -q "blocked.*gh_auth_expired"; then
pass "Worker + gh auth failure -> blocked:gh_auth_expired (t198)"
else
fail "Worker with gh auth failure should be blocked:gh_auth_expired" "Got: $eval_result"
fi

# ============================================================
# SECTION 10: Concurrent Worktrees (parallel tasks)
# ============================================================
Expand Down
31 changes: 31 additions & 0 deletions tests/test-supervisor-state-machine.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1099,6 +1099,37 @@ else
fail "Normal incomplete exit should still be clean_exit_no_signal" "Got: $eval_result"
fi

# Test: GitHub auth failure -> blocked:gh_auth_expired (t198)
# Workers that complete work but can't push due to expired gh auth should be
# blocked immediately, not retried. This is the #1 cause of wasted retries.
create_eval_task "eval-t198e" 'WORKER_STARTED task_id=eval-t198e pid=12345 timestamp=2026-02-09T03:00:00Z
{"type":"step_start","timestamp":1770606000000,"part":{"type":"step-start"}}
{"type":"text","timestamp":1770606100000,"part":{"type":"text","text":"Implementation complete. All files created and committed."}}
{"type":"tool_use","timestamp":1770606200000,"part":{"type":"tool","tool":"bash","state":{"status":"completed","input":{"command":"gh auth status"},"output":"gh auth status\nYou are not logged in to any GitHub hosts. Run gh auth login to authenticate.","metadata":{"exit":1}}}}
{"type":"text","timestamp":1770606300000,"part":{"type":"text","text":"GitHub authentication token has expired. Cannot push or create PR. The implementation is complete locally but needs gh auth login to proceed."}}
{"type":"step_finish","timestamp":1770606400000,"part":{"type":"step-finish","reason":"stop"}}
EXIT:0'
eval_result=$(sup evaluate eval-t198e --no-ai 2>&1 | grep "^Verdict:" || echo "")
if echo "$eval_result" | grep -q "blocked.*gh_auth_expired"; then
pass "Exit 0 + gh auth failure -> blocked:gh_auth_expired (not retried)"
else
fail "Exit 0 + gh auth failure should be blocked:gh_auth_expired" "Got: $eval_result"
fi

# Test: gh auth failure with different wording (try authenticating)
create_eval_task "eval-t198f" 'WORKER_STARTED task_id=eval-t198f pid=12345 timestamp=2026-02-09T03:00:00Z
{"type":"step_start","timestamp":1770606000000,"part":{"type":"step-start"}}
{"type":"tool_use","timestamp":1770606100000,"part":{"type":"tool","tool":"bash","state":{"status":"completed","input":{"command":"git push"},"output":"remote: Permission denied. try authenticating with gh auth login","metadata":{"exit":1}}}}
{"type":"text","timestamp":1770606200000,"part":{"type":"text","text":"Push failed. Need to authenticate with GitHub."}}
{"type":"step_finish","timestamp":1770606300000,"part":{"type":"step-finish","reason":"stop"}}
EXIT:0'
eval_result=$(sup evaluate eval-t198f --no-ai 2>&1 | grep "^Verdict:" || echo "")
if echo "$eval_result" | grep -q "blocked.*gh_auth_expired"; then
pass "Exit 0 + 'try authenticating' -> blocked:gh_auth_expired"
else
fail "Exit 0 + 'try authenticating' should be blocked:gh_auth_expired" "Got: $eval_result"
fi

# ============================================================
# SECTION 7: Worktree Path Integrity
# ============================================================
Expand Down
Loading