fix(reminders): remove execution cap so reminders never skip on capacity - #1839
Merged
Conversation
…ity (#1812 regression) The capacity gate (MaxConcurrentExecutions = 3) settled blocked occurrences instead of deferring them, and when the settlement ack raced with the cron reschedule it produced a bogus reminder.settlement.failed alert. Worse, it skipped real reminders (e.g. the daily vendor-price audit) when other reminders were parked on long-running executions. Every execution already has a one-hour absolute timeout and Akka.Reminders owns failure retry, so unbounded scheduling pressure on the LLM is acceptable. Remove the cap entirely - no capacity skip, no settle, no alert.
Aaronontheweb
commented
Aug 9, 2026
Aaronontheweb
left a comment
Collaborator
Author
There was a problem hiding this comment.
Now that we tolerate longer timeouts for reminders, we can afford to apply more scheduling pressure even to smaller, constrained local LLMs. the hard limit of 3 concurrent reminders + buffering we did to the rest is effectively obsolete now
Aaronontheweb
enabled auto-merge (squash)
August 9, 2026 12:34
Merged
Aaronontheweb
added a commit
that referenced
this pull request
Aug 20, 2026
* Sync delta specs for completed OpenSpec changes Apply the delta specs of 16 completed changes to the main specs. Create four new capability specs: daemon-shell-path, shell-policy-evaluator-architecture, skillserver-native-sidecar-sync, and named-model-definitions. Correct three reminder requirements against the merged code: - One-shot success removes the definition and its history. Only a poisoned one-shot is soft-deleted (PR #1821). - No execution capacity cap exists (PR #1839). The nack and ack-skip policy now covers a duplicate active occurrence and a short acknowledgement lease. - Every delivery kind now holds its envelope. ReminderDeliveryResult replaces ReminderDeliveryObserved. * Archive 16 completed OpenSpec changes Move each completed change to openspec/changes/archive/2026-08-19-<name>/. Their code is merged on dev. Also tick task 5.3 of consolidate-binding-actor-engines. PR #2005 merged that work. Leave eval-run checkboxes unticked. An evals-only gap does not block the archive.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The daily vendor-price audit was skipped this morning (Aug 9). Root cause chain:
MaxConcurrentExecutions = 3, hardcoded) was reached — three tbench/PR test reminders were parked on long executions burning their full 1h timeouts.NotFound, which tripped areminder.settlement.failedops alert for a scenario that is not an ops failure.Fix
Remove the execution cap entirely:
MaxConcurrentExecutionsconst and the capacity gate inReminderManagerActor.ExecutionAttemptTimeout), so unbounded scheduling pressure on the LLM is acceptable.Tests
Recurring_occurrence_at_capacity_is_acked_without_executionintoRecurring_occurrence_starts_even_while_other_reminders_are_running: three blockers in flight, a fourth reminder fires — asserts it is dispatched (0 skipped duplicates) and executed by the pipeline.ReminderManagerActorTestssuite passes: 35/35.Notes