Skip to content

fix(#2786): surface agent API errors when Claude Code exits 0 - #2860

Merged
waynesun09 merged 2 commits into
mainfrom
agent/2786-surface-api-errors
Jul 1, 2026
Merged

fix(#2786): surface agent API errors when Claude Code exits 0#2860
waynesun09 merged 2 commits into
mainfrom
agent/2786-surface-api-errors

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor

Claude Code exits with code 0 on API/infrastructure failures (e.g., invalid_grant, quota exhaustion) while setting is_error:true in the transcript result event. This made infrastructure errors invisible — green workflows with no PR, no comment, and no GHA error annotation.

Two changes in run.go:

  1. After exit code 0, parse the tee'd output.jsonl for
    is_error:true result events. If found, override
    lastExitCode to 1 and emit StepWarn so downstream gating
    treats the run as failed.

  2. Remove the lastExitCode != 0 gate around
    ParseTranscriptErrors so ::error:: annotations are always
    emitted, regardless of exit code.

Added ParseTranscriptFile to the TranscriptHandler interface to support checking a single JSONL file (output.jsonl) without scanning an entire directory.

Note: make lint could not run (shellcheck-py fails to download in sandbox). go vet and go test both pass.


Closes #2786

Post-script verification

  • Branch is not main/master (agent/2786-surface-api-errors)
  • Secret scan passed (gitleaks — 43388d57a168ed0b5caf97463414003677404fa2..HEAD)
  • Pre-commit hooks passed (authoritative run on runner)
  • Tests ran inside sandbox

Claude Code exits with code 0 on API/infrastructure failures
(e.g., invalid_grant, quota exhaustion) while setting
is_error:true in the transcript result event. This made
infrastructure errors invisible — green workflows with no PR,
no comment, and no GHA error annotation.

Two changes in run.go:

1. After exit code 0, parse the tee'd output.jsonl for
   is_error:true result events. If found, override
   lastExitCode to 1 and emit StepWarn so downstream gating
   treats the run as failed.

2. Remove the lastExitCode != 0 gate around
   ParseTranscriptErrors so ::error:: annotations are always
   emitted, regardless of exit code.

Added ParseTranscriptFile to the TranscriptHandler interface
to support checking a single JSONL file (output.jsonl) without
scanning an entire directory.

Note: make lint could not run (shellcheck-py fails to download
in sandbox). go vet and go test both pass.

Closes #2786
@fullsend-ai-coder
fullsend-ai-coder Bot requested a review from a team as a code owner July 1, 2026 18:08
@fullsend-ai-coder fullsend-ai-coder Bot added the ready-for-review Triggers review agent dispatch label Jul 1, 2026
@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown

Site preview

Preview: https://5728bdf3-site.fullsend-ai.workers.dev

Commit: 5f1462de4478a49fabbc65361ffac12c39e6cd37

@codecov

This comment was marked as off-topic.

@waynesun09

Copy link
Copy Markdown
Member

/review

@qodo-code-review

This comment was marked as resolved.

- Add transcriptErrorOverride flag to skip the post-script when
  Claude Code exits 0 with is_error:true in the transcript. The
  flag is scoped to transcript-error overrides only (not all
  non-zero exits) and reset each iteration.

- Simplify the three-way StepWarn branch to two-way, printing
  lastExitCode (the effective value) instead of exitCode.

Assisted-by: Claude (fix), Gemini (review), Codex (review)
Signed-off-by: Wayne Sun <gsun@redhat.com>
@github-actions

This comment was marked as resolved.

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown

E2E tests did not run

E2E tests run automatically for org/repo members and collaborators on pull requests.

For other contributors, a maintainer must add the ok-to-test label after the latest push.

See E2E testing guide for details.

@waynesun09 waynesun09 added the ok-to-test Allow e2e CI to run after maintainer review (must be re-applied after each push) label Jul 1, 2026
@waynesun09

Copy link
Copy Markdown
Member

/review

@qodo-code-review

Copy link
Copy Markdown

Persistent review updated to latest commit 5f1462d

@waynesun09 waynesun09 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Reviewed with a 5-agent review squad (Claude, Gemini, Codex), 3 fix-verify iterations.

Core logic is correct: is_error:true in the final transcript result event is a definitive session failure signal (API errors, quota exhaustion, stale OIDC tokens). No false positive risk — only fires on exit code 0 with a terminal error in the transcript.

Fixes applied on top of the original commit:

  • transcriptErrorOverride flag scoped to transcript-error overrides only (not all non-zero exits), reset each iteration
  • StepWarn prints lastExitCode (effective value) instead of exitCode
  • Post-script correctly skipped on transcript errors

codecov/patch failure is expected — runAgent is an orchestrator function that can't be unit-tested. The underlying parseTranscriptFile has thorough test coverage.

@waynesun09
waynesun09 added this pull request to the merge queue Jul 1, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 1, 2026
@waynesun09
waynesun09 added this pull request to the merge queue Jul 1, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 1, 2026
@waynesun09
waynesun09 added this pull request to the merge queue Jul 1, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 1, 2026
@waynesun09
waynesun09 added this pull request to the merge queue Jul 1, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 1, 2026
@waynesun09
waynesun09 added this pull request to the merge queue Jul 1, 2026
Merged via the queue into main with commit d0771f7 Jul 1, 2026
20 of 21 checks passed
@waynesun09
waynesun09 deleted the agent/2786-surface-api-errors branch July 1, 2026 21:47
@fullsend-ai-retro

fullsend-ai-retro Bot commented Jul 1, 2026

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 9:51 PM UTC · Completed 10:00 PM UTC
Commit: 5f1462d · View workflow run →

@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #2860 — Surface agent API errors when Claude Code exits 0

Timeline

  1. 2026-06-30 16:07 — Issue #2786 filed by waynesun09 with detailed root-cause analysis and proposed fix
  2. 2026-06-30 16:10 — First triage run 28458573867 failed
  3. 2026-07-01 17:46 — waynesun09 manually re-triggers /fs-triage
  4. 2026-07-01 17:49–17:52 — Second triage run 28536784506 succeeded, applied labels (type/bug, priority/high, component/runner, ready-to-code)
  5. 2026-07-01 17:52–18:08 — Code agent run 28537091526 succeeded, created PR #2860 with commit deacf00
  6. 2026-07-01 20:33 — waynesun09 manually runs /review (triggering qodo-code-review, not the fullsend review agent)
  7. 2026-07-01 20:34 — qodo-code-review flags three concerns: exit code reporting inconsistency, post-script gating gap, and interface ripple
  8. 2026-07-01 20:34 — waynesun09 pushes fix commit 5f1462d addressing post-script gating and StepWarn cleanup
  9. 2026-07-01 20:44 — waynesun09 approves PR after multi-agent review (Claude, Gemini, Codex)
  10. 2026-07-01 21:47 — PR merged

What went well

  • Triage quality: The second triage run produced an excellent summary that correctly identified the root cause, recommended fixes, and even proposed test cases.
  • Code agent output: The code agent implemented the two fixes proposed in the issue (remove exit-code gate on ParseTranscriptErrors, add output.jsonl parsing for is_error:true) and added three well-targeted tests.
  • Issue quality: The issue itself was exceptionally well-written with root-cause analysis, code references, and a real-world example.

What needed rework

The human pushed a follow-up commit fixing two gaps in the code agent's implementation:

  1. Post-script gating: The code agent overrode lastExitCode to 1 on transcript errors but didn't propagate this to the post-script defer block, which gates on runErr (not lastExitCode). A new transcriptErrorOverride flag was needed to skip post-script execution.
  2. StepWarn simplification: The code agent introduced a three-way if/else branch distinguishing "real non-zero exit" from "transcript error override" — the human collapsed this to a clean two-way branch using lastExitCode consistently.

Skipped proposals (covered by existing issues)

  • Review agent visibility: The fullsend review agent did not post any comment on this agent-authored PR. #2637 ("Review agent should always leave a visible trace on agent-authored PRs") already covers this gap.
  • First triage failure: The initial triage failed without a clear diagnostic. #1283 (surface triage token minting errors) and #1919 (startup_failure silently lost) cover related concerns.

Proposals filed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ok-to-test Allow e2e CI to run after maintainer review (must be re-applied after each push) ready-for-review Triggers review agent dispatch Review effort 3/5

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agent API errors silently swallowed when Claude Code exits 0 with is_error:true

1 participant