Fix workflow scheduler pagination - #5092
Conversation
Co-authored-by: Jacob Greif <greifj@gmail.com> Signed-off-by: Jacob Greif <greifj@gmail.com>
|
Production follow-up (2026-08-19): hosted native schedule fires are still not reaching newer definitions; a 120-second external recovery watchdog remains the effective launcher. This fix is therefore still needed before native cron can be trusted. The PR is currently both @jacobandjacob please rebase onto current |
What changed
(created_at, community_id, id)tuple.Why
The scheduler previously loaded only the 1,000 oldest active schedules. Once the global fleet exceeds that limit, every newer schedule is permanently invisible. Long or failed passes could also leave uncovered cron time.
This change removes the starvation boundary without unbounded definition memory. Tenant-scoped workflow IDs remain safe at page boundaries, and delayed passes attempt the latest missed occurrence without replaying every occurrence in a storm.
The hosted incident that led to this investigation still requires production schedule counts or relay logs for definitive attribution. After deployment, an automatic canary should pass before hosted schedules are considered reliable.
Validation
cargo +stable test -p buzz-db -p buzz-workflow— 94 + 156 passed at commit5198decfee48177af95b410ed1a40ea3467ee20c