fix(cron): bound per-job run history — prune old run sessions after each fire - #88331
fix(cron): bound per-job run history — prune old run sessions after each fire#88331ayushnangia wants to merge 6 commits into
Conversation
ae9d38c to
a25d317
Compare
Reviewed by reviewer-e (AI automated review). Well-bounded fix for unbounded state.db growth and BOTS/SESSIONS duplicate flooding (#88268): retention resolves env > config > default(50) with invalid values warned about, the prune's id-range scan Nit (non-blocking): cron/scheduler.py — the prune executes after every fire even when there's nothing to remove (a 1-minute job under its retention pays one wasted SELECT per tick). A cheap early-out — fetch victim ids first and skip the write transaction when empty — would keep hot jobs at zero write cost in the common case; alternatively only attempt pruning every Nth fire. |
a25d317 to
d16638b
Compare
SessionDB.prune_cron_job_runs(job_id, keep): deletes a cron job's oldest run sessions beyond the newest keep, scoped by the same [prefix, prefix_hi) id-range scan as list_cron_job_runs. Uses the delete_session cascade shape (delegate children die with the parent, branches orphaned, messages removed) so FK constraints hold; returns the count deleted. Bounds state.db growth + BOTS/SESSIONS duplicate noise from per-tick cron rows (NousResearch#88268).
After the run session is finalized (end_session, before close), prune the job's old run rows keeping cron.run_history_retention (env HERMES_CRON_RUN_HISTORY_RETENTION > config > default 50). Best-effort: a prune failure logs and never wedges bookkeeping or delivery.
- keep=3 leaves exactly the newest three runs of the target job; other jobs' runs and non-cron sessions untouched - keep=0 clears the job's history - FK cascade: messages removed with the sessions
…esearch#92133) The bare [prefix, prefix_hi) range leaks runs across jobs when one job id is an underscore-extension of another (backup vs backup_weekly): backup_weekly sorts inside backup's range, so a prune could delete another job's runs. Constrain both the victim SELECT and the keep-set to rows whose post-prefix remainder matches this run's timestamp shape (%Y%m%d_%H%M%S via GLOB), which free-form job ids cannot satisfy. Matches the list-side predicate the NousResearch#92133 PRs add; prune coverage they leave open.
Enough1122 review on NousResearch#88331: a hot job under its retention paid one write transaction per fire even when the victim set was empty. The transaction now opens only when there are rows to delete.
1d35283 to
eaa6826
Compare
Summary
Fixes #88268: a busy cron job appends one session row per tick (
cron_<jobid>_<timestamp>), growingstate.dbunbounded (the report: 473 rows from one 15-min job in 5 days) and flooding BOTS/SESSIONS with per-run duplicates.The per-run rows are the desktop run-history contract (
list_cron_job_runs), so the fix is bounded retention, not a single stable row.Changes
SessionDB.prune_cron_job_runs(job_id, keep)— deletes a job's oldest run sessions beyond the newestkeep, scoped by the same[prefix, prefix_hi)id-range scan aslist_cron_job_runs. Uses thedelete_sessioncascade shape (delegate children die with the parent, branches orphaned, messages removed, unreferenced prompts cleaned) so FK constraints hold. Returns the count deleted.end_session, beforeclose), prune keepingcron.run_history_retention(envHERMES_CRON_RUN_HISTORY_RETENTION→ config → default 50 — always a superset of the desktop's 20-row window). Best-effort: a prune failure logs and never wedges bookkeeping or delivery.Validation
tests/test_hermes_state.py -k cron: 4 passed ·tests/cron/test_cleanup_timeout.py tests/cron/test_jobs.py: 72 passed.