fix(kanban): salvage partial findings on iteration-budget exhaustion - #52511
Open
DanielMaly wants to merge 1 commit into
Open
DanielMaly wants to merge 1 commit into
DanielMaly wants to merge 1 commit into
Conversation
When a Kanban worker exhausts its iteration budget, the agent loop already asks the model for a toolless summary via _handle_max_iterations. Previously that summary was discarded: _record_task_failure was called without it, so the run row closed with summary=None and the retry worker started from scratch with no partial findings. This change forwards the model's last output as partial_summary to _record_task_failure, which writes it to the run row's summary field (via _end_run). build_worker_context already surfaces run summaries in the prior-attempts section, so the retry worker sees the partial findings automatically. Additionally, a [partial_unverified] task comment is posted containing the model's last output, so it appears in the retry worker's comment thread. The tag marks it as emergency-salvaged content from a budget-exhausted run, not a deliberate handoff. The prompt_builder KANBAN_GUIDANCE is updated with a brief iteration-budget-exhaustion paragraph so workers know the salvage mechanism exists and are encouraged to proactively kanban_comment + kanban_block before hitting the cap. Design notes: - This is the minimal DB-side salvage (Option 1 + Option 2 from the design review). It has no recursion risk (tools are stripped in the summary API call), no fabrication risk (genuine model output, clearly tagged), and no double-recording risk (status gating). - The soft-budget-warning (Option 3) and reserved-finalization-tool-turn (Option 4) approaches are complementary but deferred — the current approach is simpler and sufficient. Tests: 7 new tests (3 in test_kanban_db.py covering partial_summary plumbing for timed_out/gave_up/null cases, 4 in test_turn_finalizer_kanban_salvage.py covering the end-to-end salvage flow). 236 existing tests still pass.
Collaborator
|
Thanks for tracing the run-summary loss; current main still closes the Kanban timeout run without a summary, while retry context renders closed-run summaries. Problems
Suggested changes
Automated hermes-sweeper review. |
GravityTone
pushed a commit
to GravityTone/hermes-agent
that referenced
this pull request
Jul 18, 2026
Builds on timeout-salvage work from NousResearch#52511 and implements the preventive budget-warning contract from NousResearch#54153. Co-authored-by: Daniel Maly <maly.daniel@protonmail.com>
Open
4 tasks done
garadice
added a commit
to garadice/hermes-agent
that referenced
this pull request
Aug 25, 2026
…inst the parent task HERMES_KANBAN_TASK stays set in os.environ for in-process children running inside a dispatcher worker: delegate_task subagents, background-review forks, and cron agents fired via the cronjob tool. Those children carry their own much smaller iteration budgets (review-fork cap 16, delegation cap 50), so when a child exhausted ITS budget, _record_kanban_budget_exhausted recorded a terminal timed_out failure against the WORKER's task — advancing the consecutive-failure circuit breaker and archiving live parent work that was still running fine. Gate both call sites on is_dispatcher_owned_worker_context() (the documented single predicate for HERMES_KANBAN_* identity gates — False for delegated children and non-dispatcher-owned cron execution, from NousResearch#79657's context machinery) plus the review fork's _persist_disabled persistence-isolation marker. Zero behavior change for genuine dispatcher-owned workers. Complementary to NousResearch#52511 (partial-summary salvage on exhaustion): that PR forwards what the exhausted worker produced; this PR stops non-worker exhaustion from being attributed to the worker at all. Regression test: 5-case matrix — dispatcher-owned fires; delegated child skips; review fork skips; non-dispatcher-owned cron skips; guard truth table (mutation-checked: each guard leg independently covered).
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When a Kanban worker exhausts its iteration budget, the agent loop asks the model for a final toolless summary via
_handle_max_iterations(). Previously that summary was discarded —_record_task_failure()was called without forwarding it, so the run row closed withsummary=Noneand the retry worker started from scratch with no partial findings.This PR forwards the model's last output as
partial_summaryto_record_task_failure, which writes it to the run row'ssummaryfield via_end_run.build_worker_contextalready surfaces run summaries in the prior-attempts section, so the retry worker sees the partial findings automatically.Additionally, a
[partial_unverified]task comment is posted containing the model's last output, so it appears in the retry worker's comment thread. The tag marks it as emergency-salvaged content from a budget-exhausted run, not a deliberate handoff.Closes #52510
Root cause
In
agent/turn_finalizer.py, the budget-exhaustion path calls_record_task_failure(outcome="timed_out")but never passes thefinal_response(the model's last output from_handle_max_iterations). The_record_task_failurefunction had no parameter to accept a partial summary, and_end_run()was called withoutsummary=, leaving the run row's summary field asNULL.Changes
hermes_cli/kanban_db.py: Addpartial_summaryparameter to_record_task_failure(), forwarded to_end_run(summary=...)in both the breaker-trip (gave_up) and below-threshold paths.agent/turn_finalizer.py: Forwardfinal_responseaspartial_summaryto_record_task_failure; post a[partial_unverified]comment with the model's last output; improve the comment explaining why_record_task_failureis used instead ofkanban_block(separating the two concerns: circuit-breaker counting vs. tools being stripped).agent/prompt_builder.py: Add a brief iteration-budget-exhaustion paragraph to KANBAN_GUIDANCE so workers know the salvage mechanism exists and are encouraged to proactivelykanban_comment+kanban_blockbefore hitting the cap.test_kanban_db.pycoveringpartial_summaryplumbing (timed_out / gave_up / null cases), 4 intest_turn_finalizer_kanban_salvage.pycovering the end-to-end salvage flow.Design choices
[partial_unverified]._handle_max_iterations, so the model cannot call kanban tools during the summary API call.failure_limitnot passed fromturn_finalizer: The worker process does not have access to the dispatcher'skanban.failure_limitconfig. The_record_task_failuredefault (DEFAULT_FAILURE_LIMIT) is acceptable here — the per-taskmax_retriesoverride still takes precedence when set.Test plan
test_record_task_failure_salvages_partial_summary— verifiespartial_summarylands in run row's summary and is surfaced bybuild_worker_contexttest_record_task_failure_without_partial_summary_stays_null— backward-compatible default (existing callers unaffected)test_record_task_failure_partial_summary_on_gave_up— partial summary persists when circuit breaker tripstest_budget_exhaustion_salvages_partial_summary_to_run— end-to-end:finalize_turnforwards model output to run summarytest_budget_exhaustion_writes_partial_unverified_comment—[partial_unverified]comment is written with correct authortest_budget_exhaustion_no_summary_no_comment— no comment when model produces empty summarytest_budget_exhaustion_retry_worker_sees_partial_findings— retry worker'sbuild_worker_contextcontains both the partial summary and the comment