feat(cron): add daemon_alive() two-level heartbeat check - #70020
Conversation
Implements a two-level daemon liveness check: 1. Check PID file existence and process validity 2. Check heartbeat file freshness (configurable interval) Also includes full daemon lifecycle: _daemonize(), run_ticker(), stop_daemon(), signal handling, and atexit cleanup. Closes NousResearch#69976
teknium1
left a comment
There was a problem hiding this comment.
Thanks for addressing cron liveness. The heartbeat work on current main has moved into the provider lifecycle, so this needs substantial re-scoping before it can be safely salvaged.
Problems
cron/daemon.py:64catchesPermissionErrorviaOSError, deletes the PID file, and reports the daemon dead.os.kill(pid, 0)raisingPermissionErrorindicates that the process exists but is not signalable; the linked issue explicitly calls for this case to return true.cron/daemon.py:118callscron.scheduler.tick()directly. Current execution resolves and starts the configured scheduler provider ingateway/run.py:25491-25542;cron/scheduler_provider.py:122-159supports external providers, while the built-in provider owns heartbeat and profile-aware lifecycle behavior.- The PR contains no tests for the specified PID/heartbeat cases.
Suggested changes
- Resolve the competing standalone-daemon contract with #43864, then integrate through the selected
hermes cron/ provider lifecycle. - Preserve a live PID on
PermissionErrorand add the listed liveness and integration tests.
Automated hermes-sweeper review.
| # Check if process exists (signal 0) | ||
| os.kill(pid, 0) # windows-footgun: ok — POSIX process check; daemon is Unix-only | ||
| return True | ||
| except (ValueError, OSError, ProcessLookupError): |
There was a problem hiding this comment.
PermissionError is an OSError, but for os.kill(pid, 0) it means the PID exists and is not signalable by this user. Do not unlink the PID file or return false for that case; handle it separately as a live daemon.
| try: | ||
| while True: | ||
| try: | ||
| tick(verbose=False) |
There was a problem hiding this comment.
This bypasses the current CronScheduler provider lifecycle. Please resolve the configured provider and use its start/stop contract so external providers, profile scoping, and the built-in heartbeat behavior are not silently skipped.
Summary
Implements a two-level daemon liveness check for the cron subsystem:
Returns
Trueonly if both checks pass. Falls back gracefully when either file is missing.Also includes the complete daemon lifecycle management:
_daemonize(): Fork to background with proper PID file handlingrun_ticker(): Periodic execution loop with heartbeat updatesstop_daemon(): Clean shutdown via SIGTERMCloses #69976