Skip to content

docs(delegate): record live verification of fatal-error guardrails - #88

Merged
sungjunlee merged 1 commit into
mainfrom
docs/delegate-guardrails-live-verified
Aug 7, 2026
Merged

sungjunlee merged 1 commit into
mainfrom
docs/delegate-guardrails-live-verified

Conversation

@sungjunlee

@sungjunlee sungjunlee commented Aug 7, 2026 •

Copy link
Copy Markdown
Owner

What

Records the end-to-end verification of #87's guardrails change against a live failing route (/delegate opencode-go/glm-5.2, quota-blocked until ~08-20).

Evidence (2026-08-06, live dispatch)

  • With --print-logs --log-level ERROR, stderr surfaced Monthly usage limit reached. Resets in 13 days. ~11 s in.
  • Terminating on that line (opencode keeps running after printing it) reported dispatch_cli_error at 13 s — not dispatch_timeout minutes later, not a silent hang.
  • stdout stayed clean; extraction unaffected.

Changes

  • dispatch-guardrails.md Surface fatal errors early: appended the verified-live note to the Observed block; corrected the estimated "about 36 seconds" surfacing figure to the measured 11–36 s range across two 2026-08-06 runs, and noted terminating on the first definitive line rather than waiting for retries to exhaust.
  • markdownlint autofix on the failure-code table separator.

npm test green. No eval infrastructure added (deliberately deferred).

Summary by CodeRabbit

  • 문서
    • 치명적 오류를 조기에 감지하고 보고하는 기준을 명확히 했습니다.
    • opencode 할당량 오류가 표준 오류 출력에 나타나면 즉시 종료하고 dispatch_cli_error로 보고하도록 안내를 보완했습니다.
    • 오류 코드 표의 서식이 정리되었습니다.

Verify #87 end-to-end via a real dispatch to opencode-go/glm-5.2 (quota-blocked
until ~08-20). With --print-logs --log-level ERROR, the first 'Monthly usage
limit reached. Resets in 13 days.' line appears ~11 s in; terminating on that
line reports dispatch_cli_error at 13 s instead of spending the 30-minute
deadline. Corrects the estimated 'about 36 seconds' surfacing figure to the
measured 11-36 s range (two runs on 2026-08-06) and notes the rule is now
exercised live, not just documented and installed.
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@coderabbitai

coderabbitai Bot commented Aug 7, 2026 •

Copy link
Copy Markdown

Review Change Stack

Walkthrough

dispatch-guardrails.md에 provider quota 오류의 조기 종료 및 dispatch_cli_error 보고 조건을 명확히 기록했다. opencode의 definitive stderr 오류 검증 결과와 failure code 표 형식도 갱신했다.

Changes

Dispatch 가드레일

Layer / File(s) Summary
Quota 오류 처리 및 failure code 문서 갱신
skills/productivity/delegate/references/dispatch-guardrails.md
codex, pi, opencode의 quota 동작 관찰 결과를 추가했다. opencode의 definitive stderr 오류가 감지되면 프로세스를 조기 종료하고 reset 시간과 함께 dispatch_cli_error로 보고하도록 명확히 했다. Failure codes 표의 구분선 형식을 정규화했다.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Possibly related PRs

  • sungjunlee/skills#7: 동일한 dispatch-guardrails.md에서 opencode quota 오류 처리와 failure code 표를 갱신했다.
  • sungjunlee/skills#87: fatal 오류 가드레일과 dispatch_cli_error 동작을 직접 다뤘다.

Poem

토끼가 오류를 먼저 보았네
stderr에서 신호가 뛰어왔네
quota면 곧장 멈추고
reset 시간도 함께 적고
dispatch_cli_error로 깡충 보고하네 🐇

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed 제목은 치명적 오류 가드레일의 실시간 검증을 문서화한 변경 사항을 정확하고 간결하게 설명합니다.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/delegate-guardrails-live-verified

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@skills/productivity/delegate/references/dispatch-guardrails.md`:
- Line 22: Preserve the definitive stderr line and any parsed reset time
separately at the moment the terminal provider error is detected, instead of
relying only on the final stderr tail. Use these captured values when generating
the dispatch_cli_error report, while retaining the existing tail capture for
other diagnostic output.
- Line 23: Update the timing range in the dispatch guardrails documentation to
align with the PR’s stated observed range of 11–36 seconds, or explicitly label
10–40 seconds as a rounded range. Keep the surrounding guidance about
--print-logs, --log-level ERROR, and terminating on the first definitive line
unchanged.
- Line 23: Update the sentence describing opencode run retry handling so it is
internally consistent: remove the claim that definitive errors appear after
internal retries if termination should happen earlier, or state that each
definitive log line must be handled while retries are ongoing. Preserve the
instruction to terminate immediately upon the first definitive line.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 491a3b01-c23a-4209-b4ac-3d9dfe4e9fe3

📥 Commits

Reviewing files that changed from the base of the PR and between 294f338 and 66ffe26.

📒 Files selected for processing (1)
  • skills/productivity/delegate/references/dispatch-guardrails.md

A CLI can report a fatal provider error — quota, auth, billing — and keep running, so waiting for exit turns a known failure into a spent deadline. Observed 2026-08-06 with exhausted quotas: codex exited nonzero within seconds; pi printed `429 … quota … reset at <UTC>` and kept running; opencode printed nothing at its default log level and kept running, leaving an exhausted quota indistinguishable from a healthy silent run.
A CLI can report a fatal provider error — quota, auth, billing — and keep running, so waiting for exit turns a known failure into a spent deadline. Observed 2026-08-06 with exhausted quotas: codex exited nonzero within seconds; pi printed `429 … quota … reset at <UTC>` and kept running; opencode printed nothing at its default log level and kept running, leaving an exhausted quota indistinguishable from a healthy silent run. Verified live 2026-08-06: a `/delegate opencode-go/glm-5.2` dispatch against the same exhausted workspace surfaced `Monthly usage limit reached. Resets in 13 days.` on stderr about 11 s in, and terminating on that line reported `dispatch_cli_error` at 13 s instead of spending the 30-minute deadline.

- Treat a definitive provider error on stderr as terminal. Terminate at once and report `dispatch_cli_error` with that line and any reset time it names, rather than waiting for the process to exit or for the deadline.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

감지한 오류 줄을 별도로 보존하세요.

Line 22는 dispatch_cli_error 보고서에 definitive stderr 줄과 reset 시간을 포함하도록 요구합니다. 그러나 Line 28은 stderr의 마지막 4KiB만 보존합니다. 종료 중 추가 stderr가 출력되면 감지한 오류 줄이 tail에서 사라질 수 있습니다. 감지한 줄과 reset 시간을 별도로 저장한 뒤 보고서에 사용하세요.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@skills/productivity/delegate/references/dispatch-guardrails.md` at line 22,
Preserve the definitive stderr line and any parsed reset time separately at the
moment the terminal provider error is detected, instead of relying only on the
final stderr tail. Use these captured values when generating the
dispatch_cli_error report, while retaining the existing tail capture for other
diagnostic output.


- Treat a definitive provider error on stderr as terminal. Terminate at once and report `dispatch_cli_error` with that line and any reset time it names, rather than waiting for the process to exit or for the deadline.
- Prefer a route flag that surfaces such errors over discovering them by timeout. `opencode run` needs `--print-logs --log-level ERROR`, which surfaces quota exhaustion about 36 seconds in, after the CLI's internal retries.
- Prefer a route flag that surfaces such errors over discovering them by timeout. `opencode run` needs `--print-logs --log-level ERROR`, which surfaces quota exhaustion about 10–40 seconds in (measured 11 s and 36 s on 2026-08-06), after the CLI's internal retries — terminate at once on the first definitive line rather than waiting for the retries to exhaust.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

문서의 시간 범위를 PR 목표와 일치시키세요.

PR 목표는 두 실행 결과를 기준으로 11–36초 범위를 문서화한다고 설명합니다. 그러나 이 줄은 10–40초를 사용합니다. 실제 관측값을 기준으로 하나의 범위만 사용하거나, 반올림한 범위라는 이유를 명시하세요.

제안된 수정
- ... about 10–40 seconds in (measured 11 s and 36 s on 2026-08-06) ...
+ ... about 11–36 seconds in (measured 11 s and 36 s on 2026-08-06) ...
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- Prefer a route flag that surfaces such errors over discovering them by timeout. `opencode run` needs `--print-logs --log-level ERROR`, which surfaces quota exhaustion about 10–40 seconds in (measured 11 s and 36 s on 2026-08-06), after the CLI's internal retries — terminate at once on the first definitive line rather than waiting for the retries to exhaust.
- Prefer a route flag that surfaces such errors over discovering them by timeout. `opencode run` needs `--print-logs --log-level ERROR`, which surfaces quota exhaustion about 11–36 seconds in (measured 11 s and 36 s on 2026-08-06), after the CLI's internal retries — terminate at once on the first definitive line rather than waiting for the retries to exhaust.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@skills/productivity/delegate/references/dispatch-guardrails.md` at line 23,
Update the timing range in the dispatch guardrails documentation to align with
the PR’s stated observed range of 11–36 seconds, or explicitly label 10–40
seconds as a rounded range. Keep the surrounding guidance about --print-logs,
--log-level ERROR, and terminating on the first definitive line unchanged.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "Locate target file"
fd -a 'dispatch-guardrails\.md$' . || true

echo
echo "Target excerpt"
if [ -f skills/productivity/delegate/references/dispatch-guardrails.md ]; then
  nl -ba skills/productivity/delegate/references/dispatch-guardrails.md | sed -n '1,80p'
fi

echo
echo "Search for related guardrail terms"
rg -n "route flag|surfaces|internal retries|wait for the retries|dispatch-guardrails|opencode run|quota exhaustion" skills/productivity/delegate/references/dispatch-guardrails.md skills/productivity/delegate -S || true

Repository: sungjunlee/skills

Length of output: 313


🏁 Script executed:

#!/bin/bash
set -euo pipefail

file="skills/productivity/delegate/references/dispatch-guardrails.md"

echo "File exists: $file"
[ -f "$file" ] || exit 0

echo
echo "Target excerpt"
cat -n "$file" | sed -n '1,80p'

echo
echo "Search nearby matching terms"
rg -n "route flag|surfaces|internal retries|wait for the retries|opencode run|quota exhaustion" "$file" skills/productivity/delegate/references/dispatch-guardrails.md -S || true

Repository: sungjunlee/skills

Length of output: 6575


🌐 Web query:

opencode run --print-logs --log-level ERROR quota exhaustion internal retries after

💡 Result:

In OpenCode, internal retries for quota exhaustion (such as HTTP 429 or 402 errors indicating subscription limits) are treated as terminal events rather than transient rate-limit errors [1][2][3]. While earlier versions of the CLI erroneously treated quota exhaustion as retryable—leading to unnecessary consumption of quota windows and obscured error messages—the system has been updated to explicitly classify quota-related 429 and 402 responses as non-retryable [1][2][3]. When a quota exhaustion error occurs, the system is designed to stop the request and surface a clear, actionable message to the user, including details about the limit reached and, when available, the expected reset time [1][2]. For debugging and monitoring, you can use the following flags with the opencode run command: - --print-logs: Outputs logs directly to stderr, which can help verify if a specific error is being triggered or if a request is hanging due to retry behavior [4][5]. - --log-level ERROR: Sets the logging verbosity to only show critical errors, which is useful for isolating quota exhaustion events from general operational noise [4][5][6]. If you are experiencing persistent retry loops, it may be due to a misconfiguration in external plugins or older integrations that do not properly distinguish between transient rate limits and hard quota exhaustion [2][7][8]. Ensuring your OpenCode environment is updated to the latest version is recommended to utilize the corrected error handling logic [4][3].

Citations:


재시도 대기 관련 문장을 실제 동작과 일치하게 수정하세요.

after the CLI's internal retries는 재시도가 먼저 끝났다는 뜻이고, rather than waiting for the retries to exhaust는 재시도를 기다리지 말라는 뜻이라 서로 모순입니다. opencode run의 재시도 동작 기준에 맞게, 재시도 순서를 제거하거나 “재시도를 수행하는 동안 매 definitive line을 처리하라”처럼 일관되게 바꾸세요.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@skills/productivity/delegate/references/dispatch-guardrails.md` at line 23,
Update the sentence describing opencode run retry handling so it is internally
consistent: remove the claim that definitive errors appear after internal
retries if termination should happen earlier, or state that each definitive log
line must be handled while retries are ongoing. Preserve the instruction to
terminate immediately upon the first definitive line.

@sungjunlee
sungjunlee merged commit faf0b2d into main Aug 7, 2026
2 checks passed
@sungjunlee
sungjunlee deleted the docs/delegate-guardrails-live-verified branch August 7, 2026 04:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant