fix(ci): docs-sync bot — no errors, no timeouts, no lost PRs

[iscekic]

What this fixes

The docs-sync bot has been 100% broken since 2026-07-25 — every scheduled run dies at Prepare rolling branch (runs 30149606876, 30193436440, 30248635269) — and the last run that produced output (30122603016) silently lost 36 of 46 docs-worthy PRs. Three defects, all reproduced before this fix was written:

• A — the job crashes. Git identity was configured in upsert-pr.mjs, after prepare-branch.mjs merges origin/main. On Blacksmith runners (hostname with no domain) identity auto-detection fails, the merge aborts, the failure is misclassified as a conflict, and git merge --abort throws because no merge is in progress → exit 1, whole job dead.
• B — PRs are lost silently. kilo run can exit 0 after doing nothing (event stream ends mid-turn). execFileSync cannot recover piped stderr on exit 0, so the diagnostic was discarded; failed PRs were recorded as skipped and rendered under "Considered, no docs change needed" (30 such rows sit in docs: auto-sync with merged PRs (through 2026-07-29) #12521 today); and the processed-through watermark advanced past them unconditionally — they were never re-collected.
• C — latent timeout. 25 min per edit batch × N batches with no wall-clock budget against the 120-min job cap, plus an unbounded kilo run in Fix verify failures. A timeout-minutes cancellation discards completed doc edits with no PR and no watermark advance.

The fix — one invariant

The watermark never moves past a PR that has no terminal outcome (terminal := action !== "pending"; a deliberate agent-written skipped is terminal). processed-through is now when every worthy PR is covered; otherwise the earliest uncovered PR's merged_at − 1 ms, so collect.mjs's existing merged:>=since query re-collects the uncovered PRs on the next run. No retry queue, no attempt counters, no new state anywhere.

Around that invariant:

• Git identity is configured once in the workflow before any git call; merge failures are classified (genuine conflict → the existing soft dated-branch fallback is preserved on purpose; anything else → loud failure).
• kilo run goes through one spawnSync wrapper so stderr is recoverable even on exit 0; exit 0 is no longer trusted — a missing summary file or unparseable output is a failure logged with the stderr tail and exit code on every attempt; 3 attempts with 60s/300s backoff.
• Failed/deferred edit batches and every non-verdict triage entry (full-chunk failure, partial-chunk backfill, budget-skipped chunk) are marked pending, never disguised as "no docs needed". The PR body gains a Pending — will retry section (informational only, 60-row display cap).
• Hard wall-clock budgets: triage 35 min, edit 50 min (15/batch), fix-verify 10 min — worst case 102 min against the 120-min cap, an 18-minute reserve, with deferred PRs left uncovered and counted. Update docs is continue-on-error so a failed edit step can no longer discard written edits.
• The 14-day watermark cap announces itself (::warning::, timestamps only) when it abandons PRs, and no longer clamps an explicit since override (a hidden expiry would have started eating the recovery set on 2026-08-01).
• A since-override replay that strands PRs (uncovered + no body rewrite) emits an explicit ::warning:: from both producers that can see it (triage.mjs when worthy == 0, upsert-pr.mjs otherwise) — the scheduled path needs no warning because the untouched marker already holds the window.
• A branch-dispatched run now restores .github/docs-sync from the dispatched ref after prepare-branch switches branches — without this, a dispatch from a fix branch would silently execute main's scripts (git restore --source=, not checkout --, so the restored files can't be swept into the docs commit).
• New selftest.mjs (plain node:assert, no network/LLM/dependencies) proves all of the above offline: merge classification, stderr-on-exit-0 recovery, the watermark invariant six ways plus the unresolvable-merged_at fallback, routing and body round-trip with marker-forgery guards, the legacy-row scrub, re-collection arithmetic, budgets with a call-recording stub, both cap arms, and the triage-side summary + replay warning (including the false-positive arm). It runs as a read-only, secretless job on PRs that touch the bot, and as the first step of the daily job.

Two deliberate simpler-shape decisions

1. The retry queue was deleted, not built. Earlier revisions carried a machine-read "pending queue" in the PR body; three review rounds found four independent defects inside it (unreachable gate, empty-PR creation, cap-overflow loss, draft-state corruption). It was replaced by the single watermark invariant above, reusing the existing merged:>=since collection query and the existing 14-day cap. A held-back watermark has none of those failure modes because it adds no mechanism. The Pending table is purely informational; the watermark, not the table, is the retry mechanism.
2. The CLI exit-code fix was cut to its own follow-up PR. kilo run genuinely reports success after an incomplete run — two distinct paths: the event stream ending before session.status: idle (packages/opencode/src/cli/cmd/run.ts:731,833-838), and a mid-stream provider error that never sets error because session.error is ignored when its session id doesn't match (run.ts:814); packages/opencode/test/cli/run/run-process.test.ts:49-65 locks today's behaviour in deliberately, so flipping it is a debatable repo-wide contract change requiring its own announced decision. It also cannot be observed by this workflow (the bot installs the published @kilocode/cli), and the bot is broken now — bundling the two risks stalling the urgent repair. Nothing here depends on it: the bot no longer trusts exit codes at all.

Verification

• node .github/docs-sync/selftest.mjs — 8 case groups, all green (also runs as the docs-sync / selftest check on this PR).
• bun run script/check-workflows.ts — ok; oxlint .github/docs-sync — 0 errors.
• End-to-end (run from this branch, exactly what the daily schedule does):
  • dry_run=true — green: selftest ran inside sync (branch-code proof), watermark resolved from docs: auto-sync with merged PRs (through 2026-07-29) #12521's marker, 60 PRs collected, triage 60/8 worthy/0 pending.
  • since=2026-07-17 replay — success in 76 min (< 120 cap): Prepare rolling branch succeeded (Defect A's crash step), Restore docs-sync scripts ran, Upsert rolling PR ran. The upstream kilo run stream failures recurred (all 11 edit batches failed) — and this time every failure logged its stderr tail and exit code, the 51 edit failures + 54 triage-pending PRs were recorded pending (never "skipped"), the partial doc edits the dying agents had already written were committed (13 files) instead of discarded, and the watermark was held back to 2026-07-17T01:19:29.999Z (earliest uncovered merged_at − 1 ms) so the next scheduled run re-collects all 105 uncovered PRs automatically. The 30 misclassified "Considered" rows in docs: auto-sync with merged PRs (through 2026-07-29) #12521 were scrubbed; its body now carries a Pending — will retry section naming each PR with its real cause. The CLI-side root cause (stream ends mid-turn while the CLI still exits 0) is the deliberate follow-up described above — the bot now survives it and reports it.

Out of scope (recorded so they are not silently forgotten): normalising inconsistent Docs change paths in the PR body; recovering rows already lost to ROW_CAP (only the 30 surviving rows are recovered, via the replay); the per-PR fetch-failure drop path in collect.mjs (already visible via ::warning:: + counter); the zero-worthy-day marker advance (pre-existing, practically unreachable).

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

The incremental change (commit eeb19d905b) scopes the pull_request trigger to its own docs-sync-pr-{number} concurrency group, separate from the scheduled docs-sync group used by the 120-minute daily sync job. This resolves the previously flagged WARNING about PR selftest runs queuing behind the long-running scheduled job.

Files Reviewed (1 file)

• .github/workflows/docs-sync.yml

Previous Review Summary (commit 3316b3b)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 3316b3b)

Status: 1 Issue Found | Recommendation: Address before merge

Overview

Severity	Count
CRITICAL	0
WARNING	1
SUGGESTION	0

Issue Details (click to expand)

WARNING

File	Line	Issue
.github/workflows/docs-sync.yml	29	New pull_request trigger shares the workflow-level concurrency: group: docs-sync slot with the up-to-120-minute scheduled sync job, which can queue PR selftest checks (or the next scheduled run) for a long time

The core fix logic (watermark-holds-back invariant, git-identity-before-merge, spawnSync-based stderr recovery, wall-clock budgets) reads correctly and is well covered by the new offline selftest.mjs. The retry/backoff, computeUncovered/computeProcessedThrough/routeRows routing, and marker-forgery stripping in upsert-pr.mjs all match the described invariants and their test cases.

Files Reviewed (8 files)

• .github/docs-sync/edit.mjs
• .github/docs-sync/lib.mjs
• .github/docs-sync/prepare-branch.mjs
• .github/docs-sync/selftest.mjs
• .github/docs-sync/triage.mjs
• .github/docs-sync/upsert-pr.mjs
• .github/docs-sync/watermark.mjs
• .github/workflows/docs-sync.yml

Fix these issues in Kilo Cloud

---

_{Reviewed by claude-sonnet-5 · Input: 26 · Output: 3.2K · Cached: 547.7K}

_{Review guidance: REVIEW.md from base branch main}