Conversation
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 7 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
Comment |
…n was cut off mid-verification Agentflare-Agent: claude-code Agentflare-Branch: task/68 Agentflare-Item: 68
…t jobs respawn_from_stale exits the process without waiting for any job the WorkerPool may have running on another thread. That looks like it could reintroduce the exact silent-failure shape item #68 exists to close, but it doesn't: dashboard::server's reconcile_orphaned_jobs already makes an abruptly-dead daemon's in-flight jobs fail loud and release their claim (item #40's crash/kill-9/OOM case), and this self-triggered exit is just another instance of that same case. Agentflare-Agent: claude-code Agentflare-Branch: task/68 Agentflare-Item: 68
Fixes the fmt CI gate on this PR. Agentflare-Agent: claude-code Agentflare-Branch: task/68 Agentflare-Item: 68
execute_work's "claim succeeded but no worktree was created" and "failed to chdir into <worktree>" paths were structural git-state failures (e.g. a stale prunable worktree registration, confirmed live for items #465/#466) that retry identically every time, but went through the normal max_retries backoff budget before the terminal- failure recovery hook could swap the item off `dispatched`. Add a `fatal` flag to JobFailure/WorkOutcome that Queue::fail uses to skip straight to terminal `state=failed` regardless of retries remaining, and set it on those two setup-failure branches. Agent-run failures are unaffected and keep the existing retry/cooldown behavior. dispatch_item's ready-for-work/dispatched label-flip timing is untouched. Agentflare-Agent: claude-code Agentflare-Branch: task/467-fail-fast-on-structural-worktree-setup-f Agentflare-Item: 467
Summary
Adds a background watchdog so a long-running
agentflare daemonself-restarts once the on-disk binary it was launched from changes, instead of silently continuing to execute stale in-process job logic forever.Why
This is the second confirmed occurrence of the same root cause:
No such file or directorybecausecurrent_exe()resolved to/home/avihs/.cargo/bin/agentflare (deleted)after a rebuild replaced the binary the daemon started from.Because work dispatch is in-process, a rebuild/merge no longer makes dispatch fail loudly — the daemon can run arbitrarily stale logic indefinitely with zero self-awareness, and failures are silent and hard to notice, not loud crashes.
What changed
src/daemon.rs:BinarySnapshotcaptures the on-disk binary's path/mtime/size viacurrent_exe()at daemon startup.is_stale()is a cheapstat()-based comparison (mtime/size changed, or the file is gone).respawn_from_stale()cleans up pid/lock files first (so the replacement's ownis_daemon_running()gate doesn't mistake the exiting process for a live one), spawns a replacement daemon via the sameserve --_foreground-daemoninvocationstart_daemon/the installed systemd/launchd units use, and exits. 3 unit tests coveris_stalefor unchanged/replaced/deleted files.src/dashboard/server.rs:spawn_binary_staleness_watchdog()ticks everyBINARY_STALENESS_CHECK_INTERVAL(60s) and callsrespawn_from_staleonce the snapshot goes stale. Wired intorun()right after the worker pool starts.respawn_from_staledeliberately does not wait for jobs theWorkerPoolmay have running. This is documented as safe rather than a new gap —reconcile_orphaned_jobs(item feat: add eyre + color-eyre for rich error reporting #40) already makes an abruptly-dead daemon's in-flight jobs fail loud and release their claim, and a self-triggered respawn is just another instance of that same crash/kill-9 case it was built for.Test plan
cargo build --bin agentflare— cleancargo test --bin agentflare— 1263 passed; 1 pre-existing unrelated failure (mcp_prompts::tests::optimize_review_returns_full_skill_body, root-caused to a sandbox artifact — a read-only~/.local/statemount rejectingflare-code's active-mode file write, unrelated to this change and reproducible on unmodified files). All 3 newbinary_snapshot_testspass.AGENTFLARE_HOME_OVERRIDE/XDG_RUNTIME_DIRsandbox, replaced the on-disk binary via rename (same shape asupdate::swap::replace_binary), and confirmed the watchdog detected the change and calledrespawn_from_stalewithin one 60s tick — old process pid gone, pid file updated to the new process, cleanup performed as designed.Closes #68