fix(kanban): verify created artifacts before marking task done (#25288) - #25328
Closed
xxxigm wants to merge 4 commits into
Closed
fix(kanban): verify created artifacts before marking task done (#25288)#25328xxxigm wants to merge 4 commits into
xxxigm wants to merge 4 commits into
Conversation
First call with the key mints a job; subsequent calls return the existing job unchanged with response.reused=true. Prevents the duplicate-cron half of NousResearch#25288 (agent retries created N copies). Also adds find_jobs() helper used by the kanban verification gate in the next commit.
…esearch#25288) complete_task gains a created_artifacts gate parallel to the existing created_cards gate. Each entry is {kind, id}; kind="cron" is verified via cron.jobs.get_job(id). Phantom entries raise HallucinatedArtifactsError + audit event; task stays in-flight so the worker can fix the claim and retry. ARTIFACT_VERIFIERS registry keeps the gate forward-compatible — unknown kinds land on an advisory bucket and never block. kanban_complete tool gets the matching schema field + a structured retry-friendly tool_error following the NousResearch#22923 contract.
…h#25288) KANBAN_GUIDANCE rules 5a (verify artifacts) + 5b (idempotency_key) plus a Do-NOT entry. SKILL adds GOOD/BAD examples mirroring the existing created_cards section. Kept compact (KANBAN_GUIDANCE 4389 chars) so cached prompts pay minimal extra tokens.
…#25288) 33 new tests cover the gate truth-table, complete_task end-to-end, the kanban_complete tool error shape, the cronjob idempotency plumbing, and the KANBAN_GUIDANCE teaching anchors. Bumps the prompt-size guard from 4096 to 5120 to fit the new rules (pre-fix prompt was already at ~3.85 KB).
xxxigm
force-pushed
the
fix/kanban-verify-artifact-completion-25288
branch
from
May 14, 2026 01:05
af128f4 to
b180cd3
Compare
Contributor
|
Thanks @xxxigm — closing this one. The artifact-verification gate + cron idempotency_key (1141 LOC) is substantive policy territory — similar in spirit to #21925 (verifier evidence) and #25356 (QC review), both of which I also closed for design-discussion reasons. Three different proposed designs for the same concern (gating completion on quality) competing in our PR queue is a signal that the design hasn't been chosen yet. If you want to revisit, please open an issue first comparing the three approaches. Appreciate the careful work. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Stops Kanban workers from marking tasks as
donewhile the cron job (or other artifact) they claim to have created does not actually exist. Fixes the silent-success path described in #25288.Three small additions wired together:
Kernel gate (
hermes_cli/kanban_db.py) —complete_taskgains acreated_artifacts=[{kind, id}]parameter parallel to the existingcreated_cardsgate.kind="cron"is verified viacron.jobs.get_job(id); phantom entries raiseHallucinatedArtifactsError+ audit event, task staysrunningso the worker can fix the claim and retry. Unknown kinds land on an advisory bucket so plugins can ship new kinds ahead of their verifier.Cron idempotency (
cron/jobs.py) —create_jobgains an optionalidempotency_key; same key returns the existing job withreused=trueinstead of duplicating. Newfind_jobs()lookup helper used by the kernel verifier.Tool surface + agent teaching —
kanban_completeschema addscreated_artifactswith a structured retry-friendly tool_error on rejection.cronjobschema addsidempotency_key.KANBAN_GUIDANCErules 5a + 5b (compressed, +530 chars over baseline) and the kanban-worker SKILL pick up the new pattern with GOOD/BAD examples.Related Issue
Fixes #25288
Type of Change
Changes Made
cron/jobs.py—idempotency_keyoncreate_job+ newfind_jobs(idempotency_key=…, name=…)helper.hermes_cli/kanban_db.py— newHallucinatedArtifactsError,ARTIFACT_VERIFIERSregistry (today:cron),_normalize_artifacts,_verify_created_artifacts, pluscreated_artifacts=…oncomplete_task.tools/kanban_tools.py—kanban_completeschema + handler threadcreated_artifactsthrough and translate the new error into a structured tool_error (still in-flight, retry hint, three-option decision tree).tools/cronjob_tools.py—cronjobaction='create' threadsidempotency_keythrough; response now includesreused: bool.agent/prompt_builder.py—KANBAN_GUIDANCErules 5a/5b + Do-NOT entry; compressed to fit cached-prompt budget.tests/tools/test_kanban_artifact_gate.py— 33 new tests covering the gate truth-table,complete_taskend-to-end (incl. exact [BUG] Agent marks Kanban task as DONE but cron was never created #25288 reproduction), tool error shape, cron idempotency, and prompt teaching anchors.tests/tools/test_kanban_tools.py— bumpedKANBAN_GUIDANCEsize guard from 4096 → 5120 with an explanatory docstring.skills/devops/kanban-worker/SKILL.md— two new sections mirroring the existingcreated_cardspattern.Backwards compatible: every new field defaults to
None, every existing call site keeps working, no schema migration.How to Test
End-to-end behaviour after the fix:
The agent's correct response (now taught by the prompt + SKILL):
A retry of the cron creation with the same
idempotency_keyreturnsreused: trueinstead of duplicating the job — fixes the second half of the post-mortem.Checklist
feat(cron):,fix(kanban):,docs(kanban):,test(kanban):)KANBAN_GUIDANCE,kanban-workerSKILL, and tool schema descriptions