Skip to content

fix(kanban): restore completion audit dispatch - #61

Merged
sahilm-ai merged 7 commits into
mainfrom
fix/restore-kanban-completion-audit
Jul 15, 2026
Merged

sahilm-ai merged 7 commits into
mainfrom
fix/restore-kanban-completion-audit

Conversation

@sahilm-ai

@sahilm-ai sahilm-ai commented Jul 15, 2026 •

Copy link
Copy Markdown
Collaborator

Diagnosis

The fork rebase replayed the completion-audit commit as e0042ea6de, but omitted its core kanban_db.py hunks. The canonical earlier commit 5f65c7af92 contains the full implementation and is not an ancestor of the rebased fork main.

Fix

  • Restore legacy completion_audit_at migration and sparse index.
  • Restore skip-review detection plus atomic audit claim and completion lifecycle helpers.
  • Restore completion-audit dispatch, including skill injection, rearming after workspace/spawn failures, and audit reporting.
  • Preserve the current fork's crash-breaker, quarantine, worktree, and review code paths.

Verification

  • scripts/run_tests.sh tests/hermes_cli/test_kanban_completion_audit.py tests/hermes_cli/test_kanban_dispatcher_crash_breaker.py tests/hermes_cli/test_kanban_human_review.py tests/hermes_cli/test_kanban_merging.py -q — 85 passed.
  • scripts/run_tests.sh tests/hermes_cli/test_kanban_*.py -q — 882 passed; one pre-existing failure in test_default_spawn_injects_hermes_max_iterations, reproduced unchanged on myfork/main (11 passed, 1 failed).
  • uv run ruff check hermes_cli/kanban_db.py — passed.
  • uv run ty check hermes_cli/kanban_db.py reports 9 existing diagnostics outside this change.

Closes the fork-main completion-audit CI regression.

Summary by CodeRabbit

  • New Features

    • Automated completion audits for tasks marked as done, processed by dedicated workers and tracked with audit run outcomes.
    • Audits are conditionally scheduled; tasks can opt out with a skip-review: directive.
  • Bug Fixes

    • Improved recovery for active audit workers (heartbeat, timeout, stuck, crash) without blocking the parent task.
    • Enhanced dispatch failure handling to close stalled audit attempts and re-queue cleanly.
    • Updated garbage collection to preserve scratch/worktree workspaces while an audit is queued.
  • Tests

    • Expanded coverage for audit recovery and GC safety across all supported failure modes.

@coderabbitai

coderabbitai Bot commented Jul 15, 2026 •

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@sahilm-ti, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 27 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 7a33c636-4ed3-4ab6-bdba-cbe075d0b958

📥 Commits

Reviewing files that changed from the base of the PR and between a41204f and 6ce3c56.

📒 Files selected for processing (2)
  • hermes_cli/kanban_db.py
  • tests/hermes_cli/test_kanban_completion_audit.py
📝 Walkthrough

Walkthrough

Adds completion audits for done tasks, including database scheduling fields, skip-review parsing, atomic claim/complete APIs, task-run events, dispatcher support, worker recovery, failure requeueing, and workspace-retention rules.

Changes

Completion audit workflow

Layer / File(s) Summary
Audit scheduling data and directives
hermes_cli/kanban_db.py
Adds the completion_audit_at column and partial index, parses skip-review: directives, and treats active audit runs as worker activity.
Audit claim and completion APIs
hermes_cli/kanban_db.py
Adds atomic audit claiming, completion, rearming, and recovery while preserving each parent task’s done status.
Completion audit worker dispatch
hermes_cli/kanban_db.py
Extends dispatch_once to claim due tasks, resolve workspaces, validate skills, spawn workers, record audits, and requeue failures.
Audit recovery and workspace retention
hermes_cli/kanban_db.py, tests/hermes_cli/test_kanban_completion_audit.py
Extends heartbeat and failure detection for audit workers and prevents garbage collection of workspaces with queued audits, with recovery and dispatch-failure coverage.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant dispatch_once
  participant kanban_db
  participant AuditWorker
  participant WorkspaceGC
  dispatch_once->>kanban_db: Find due done task
  dispatch_once->>kanban_db: Claim completion audit
  dispatch_once->>AuditWorker: Spawn sdlc-completion-audit worker
  AuditWorker->>kanban_db: Heartbeat or complete audit
  kanban_db->>kanban_db: Requeue stale, timed-out, stuck, or crashed audit
  WorkspaceGC->>kanban_db: Exclude tasks with queued completion audits
Loading

Possibly related PRs

Suggested reviewers: teknium1, sahilm-ti

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely reflects the main change: restoring completion audit dispatch in kanban.
Docstring Coverage ✅ Passed Docstring coverage is 90.91% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/restore-kanban-completion-audit

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@hermes_cli/kanban_db.py`:
- Around line 4298-4365: Update claim_completion_audit_task and its three
failure/rearm branches to centralize rearming through a helper that calls
_end_run() for the active audit run before clearing claim_lock/claim_expires and
restoring completion_audit_at. Ensure the task’s current_run_id and run status
are closed consistently so crashed audits can be requeued without leaving an
open task_runs row.
- Around line 9908-9944: Update the task-selection query in
gc_scratch_workspaces to require completion_audit_at IS NULL, preventing queued
completion audits from being deleted; apply the same guard in
gc_worktree_workspaces when its 24-hour cleanup query can remove audit-pending
tasks.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ed725696-b72b-44c7-9245-445c7d7ec7ab

📥 Commits

Reviewing files that changed from the base of the PR and between 25b478d and a326ef5.

📒 Files selected for processing (1)
  • hermes_cli/kanban_db.py

Comment thread hermes_cli/kanban_db.py
Comment thread hermes_cli/kanban_db.py
@sahilm-ti

Copy link
Copy Markdown
Owner

auto-review: changes requested.

Matrix checks (U1-U5, C1-C5)

  • U1 in-scope files: PASS — only hermes_cli/kanban_db.py, the specified restore surface.
  • U2 out-of-scope deletions: PASS.
  • U3 no secrets: PASS.
  • U4 AC coverage: PARTIAL — normal-path completion audits are restored, but failure/retry lifecycle and queued-workspace retention are not safe.
  • U5 mergeable: PASS — CLEAN.
  • C1 CI green: PASS — required GitHub Actions checks passed.
  • C2 type discipline: PASS — no new semantic escape hatch (the Any import pre-existed).
  • C3 lint clean: PASS — CI ruff + ty diff passed.
  • C4 tests touched: FAIL — source changed but this PR does not modify a test; add regression coverage for the failure paths below.
  • C5 worker identity: PASS — a326ef5 author and committer are Sahil (AI) / 266772320+sahilm-ai@users.noreply.github.com.

Code-quality judgment (role-reviewer)

  • [Major] Audit-run lifecycle: hermes_cli/kanban_db.py:9953-9960, 9973-9978, 9997-10002 rearm an audit after failure but do not call _end_run(). The claim at 4298-4365 created a running task_runs row and set current_run_id; the controlled forced-workspace-failure reproduction left the task done, requeued, and pointing at an open task_runs row (status='running', outcome=NULL, ended_at=NULL). Centralize rearming so it closes the audit run before releasing/requeuing it, and test each rearm path.
  • [Minor] Queued-audit workspace loss: hermes_cli/kanban_db.py:5033-5039 and 5085-5092 collect done tasks solely by status and unlocked claim. A queued completion audit intentionally has status='done', claim_lock IS NULL, and completion_audit_at IS NOT NULL, so scratch GC can delete its workspace before the audit dispatcher claims it (and worktree GC can do so after its grace period). Exclude completion_audit_at IS NOT NULL in both queries and cover it with a regression test.

Findings are mechanical (matrix) or judgment-based (role-reviewer). If a finding looks wrong, leave a counter-comment on the kanban task and Sahil will adjudicate on human-review.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
tests/hermes_cli/test_kanban_completion_audit.py (2)

340-352: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Avoid hardcoded temporary paths.

To align with standard pytest practices and resolve the static analysis warning for insecure temporary files, consider using the tmp_path fixture instead of hardcoding /tmp.

♻️ Proposed refactor
-def test_queued_completion_audit_worktree_survives_gc(kanban_home, monkeypatch):
+def test_queued_completion_audit_worktree_survives_gc(kanban_home, monkeypatch, tmp_path):
     """Worktree GC also keeps a done workspace while its audit is queued."""
     removed = []
     monkeypatch.setattr(kb, "remove_worktree", lambda task_id, path: removed.append(task_id))
 
     with kb.connect() as conn:
         task_id = kb.create_task(
             conn,
             title="audit before worktree cleanup",
             assignee="alice",
             workspace_kind="worktree",
-            workspace_path=str(Path("/tmp") / "audit-worktree"),
+            workspace_path=str(tmp_path / "audit-worktree"),
         )
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/hermes_cli/test_kanban_completion_audit.py` around lines 340 - 352,
Update test_queued_completion_audit_worktree_survives_gc to accept pytest’s
tmp_path fixture and derive workspace_path from it instead of constructing a
hardcoded /tmp path; preserve the existing task setup and worktree cleanup
assertions.

Source: Linters/SAST tools


285-306: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Improve readability and avoid hardcoded temporary paths.

Consider using a nested function instead of the lambda generator throw() trick for better readability. Additionally, using the standard tmp_path pytest fixture instead of hardcoding /tmp avoids static analysis warnings for insecure temporary files.

♻️ Proposed refactor
-@pytest.mark.parametrize("failure", ["workspace", "spawn"])
-def test_dispatch_completion_audit_failure_closes_run_and_requeues(
-    kanban_home, all_assignees_spawnable, monkeypatch, failure
-):
+@pytest.mark.parametrize("failure", ["workspace", "spawn"])
+def test_dispatch_completion_audit_failure_closes_run_and_requeues(
+    kanban_home, all_assignees_spawnable, monkeypatch, failure, tmp_path
+):
     """A failed audit dispatch leaves no running run behind before requeueing."""
     def failing_spawn(task, workspace, board=None):
         raise RuntimeError("audit worker unavailable")
 
     with kb.connect() as conn:
         task_id = kb.create_task(conn, title="retry audit", assignee="alice")
         _complete_task_no_pr(conn, task_id)
         if failure == "workspace":
-            monkeypatch.setattr(
-                kb,
-                "resolve_workspace",
-                lambda task, board=None: (_ for _ in ()).throw(
-                    RuntimeError("workspace unavailable")
-                ),
-            )
+            def failing_resolve(task, board=None):
+                raise RuntimeError("workspace unavailable")
+
+            monkeypatch.setattr(kb, "resolve_workspace", failing_resolve)
         else:
-            monkeypatch.setattr(kb, "resolve_workspace", lambda task, board=None: Path("/tmp"))
+            monkeypatch.setattr(kb, "resolve_workspace", lambda task, board=None: tmp_path)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/hermes_cli/test_kanban_completion_audit.py` around lines 285 - 306,
Improve test_dispatch_completion_audit_failure_closes_run_and_requeues by
replacing the generator-based lambda used for the workspace failure with a
nested function that raises RuntimeError directly, and add pytest’s tmp_path
fixture to use for the successful resolve_workspace mock instead of the
hardcoded /tmp path.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/hermes_cli/test_kanban_completion_audit.py`:
- Around line 340-352: Update test_queued_completion_audit_worktree_survives_gc
to accept pytest’s tmp_path fixture and derive workspace_path from it instead of
constructing a hardcoded /tmp path; preserve the existing task setup and
worktree cleanup assertions.
- Around line 285-306: Improve
test_dispatch_completion_audit_failure_closes_run_and_requeues by replacing the
generator-based lambda used for the workspace failure with a nested function
that raises RuntimeError directly, and add pytest’s tmp_path fixture to use for
the successful resolve_workspace mock instead of the hardcoded /tmp path.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 401afaa0-92ea-429c-a3f3-5b71f1c5373e

📥 Commits

Reviewing files that changed from the base of the PR and between a326ef5 and fa86df0.

📒 Files selected for processing (2)
  • hermes_cli/kanban_db.py
  • tests/hermes_cli/test_kanban_completion_audit.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • hermes_cli/kanban_db.py

@sahilm-ai

Copy link
Copy Markdown
Collaborator Author

Completion-audit follow-up

Addresses the auto-review findings from the prior head:

  • A shared _rearm_completion_audit closes the claimed task_runs record with spawn_failed, clears current_run_id, then requeues the done audit task.
  • All dispatcher rearm branches use it: workspace resolution, missing skill validation, and worker spawn failure.
  • Scratch and worktree GC now exclude tasks with a queued completion audit.
  • Regression coverage exercises workspace/spawn rearm failures and both GC paths.

Validation:

  • scripts/run_tests.sh tests/hermes_cli/test_kanban_completion_audit.py -q → 22 passed.
  • scripts/run_tests.sh for crash-breaker, human-review, and merger suites → 67 passed.
  • Full Kanban shard → 886 passed; one existing unrelated failure remains in test_default_spawn_injects_hermes_max_iterations.
  • CI run 29425854474: all eight Python slices and ruff+ty passed; its only failure was the e2e job before tests because the runner's curl download of ripgrep failed with Recv failure: Connection reset by peer.

CodeRabbit’s two substantive lifecycle/GC comments are addressed in fa86df0; fixture-isolation nits are addressed in 2274703.

@sahilm-ti

Copy link
Copy Markdown
Owner

auto-review: changes requested.

Matrix checks (U1–U5, C1–C5)

  • U1 in-scope files: PASS — only the completion-audit implementation and its regressions.
  • U2 out-of-scope deletions: PASS.
  • U3 no secrets: PASS.
  • U4 AC coverage: FAIL — a completion-audit worker that dies after being successfully spawned cannot be recovered or retried.
  • U5 mergeable: PASS — CLEAN.
  • C1 CI green: PASS — GitHub Actions run 29426254315 has all required checks green.
  • C2 type-discipline: PASS — Callable is an ordinary annotation; no new type escape hatch.
  • C3 lint clean: PASS — CI ruff + ty diff passed.
  • C4 tests touched: PASS.
  • C5 worker identity: PASS — all three commits use Sahil (AI) / 266772320+sahilm-ai@users.noreply.github.com.

Code-quality judgment (role-reviewer)

  • [Major] Audit worker crash lifecycle: hermes_cli/kanban_db.py:4298-4365 claims an audit while retaining status='done'; hermes_cli/kanban_db.py:3893-3899 limits stale-claim recovery to status='running', and the runtime/heartbeat recovery paths are likewise running-only (7861-7869, 8004-8012, 8161-8169). A forced expired audit claim reproduced reclaimed=0, status='done', a non-null claim_lock and current_run_id, plus an unended task_runs row. The new pre-spawn rearm helper does not cover a worker that crashes after _set_worker_pid() succeeds, so that audit remains permanently stuck. Add audit-aware crash/TTL recovery that closes the active audit run and requeues completion_audit_at, then cover the post-spawn worker-death path.

Findings are mechanical (matrix) or judgment-based (role-reviewer). If a finding looks wrong, leave a counter-comment on the kanban task and Sahil will adjudicate on human-review.

@sahilm-ti

Copy link
Copy Markdown
Owner

auto-review: changes requested.

Matrix checks (U1–U5, C1–C5)

  • U1 in-scope files: PASS — hermes_cli/kanban_db.py plus its focused completion-audit regression file.
  • U2 out-of-scope deletions: PASS — no deletions.
  • U3 no secrets: PASS.
  • U4 AC coverage: PASS for the original completion-audit restore and the prior audit-rearm/GC findings.
  • U5 mergeable: PASS — CLEAN.
  • C1 CI green: PASS — all required checks in run 29427640278 passed.
  • C2 type-discipline: PASS — no new ignore/cast/Any escape hatch.
  • C3 lint clean: PASS — CI ruff + ty diff passed.
  • C4 tests touched: PASS — audit regression tests added.
  • C5 worker identity: PASS — all four PR commits use 266772320+sahilm-ai@users.noreply.github.com.

Code-quality judgment (role-reviewer)

  • [Major] Active completion audits are still invisible to three worker-lifecycle controls: hermes_cli/kanban_db.py:3844-3857, hermes_cli/kanban_db.py:7920-7924, hermes_cli/kanban_db.py:8215-8224, and hermes_cli/kanban_db.py:8869-8872.
    • Evidence: an audit claim deliberately keeps tasks.status = 'done' (claim_completion_audit_task, hermes_cli/kanban_db.py:4324-4331), but heartbeat_claim only accepts _WORKER_ACTIVE_STATUS_SQL; runtime-cap, stuck-worker, and immediate crash queries all require status = 'running'.
    • Why: a successfully spawned audit worker cannot heartbeat; if it stays PID-alive, release_stale_claims extends its expired claim (hermes_cli/kanban_db.py:3913-3953) while its heartbeat remains NULL, so neither stuck detection nor max-runtime enforcement can terminate/requeue it. A dead audit worker is also missed by the immediate crash detector and waits for TTL recovery. This leaves a live audit able to hold a run indefinitely.
    • Fix: model a claimed completion audit as an active worker state in heartbeat, max-runtime, stuck, and crash recovery; each recovery must use the existing audit-aware close/requeue helper so the task remains done. Add regressions for audit heartbeat plus post-spawn timeout/stuck/crash recovery.

Focused verification: scripts/run_tests.sh tests/hermes_cli/test_kanban_completion_audit.py -q → 23 passed.


Findings are mechanical (matrix) or judgment-based (role-reviewer). If a finding looks wrong, leave a counter-comment on the kanban task and Sahil will adjudicate on human-review.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
tests/hermes_cli/test_kanban_completion_audit.py (1)

355-381: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Assert that each recovery actually requeues the audit.

These tests pass if claims/runs are cleared but completion_audit_at is accidentally left NULL.

  • tests/hermes_cli/test_kanban_completion_audit.py#L355-L381: assert completion_audit_at IS NOT NULL after timeout recovery.
  • tests/hermes_cli/test_kanban_completion_audit.py#L384-L419: assert it after stuck-worker recovery.
  • tests/hermes_cli/test_kanban_completion_audit.py#L422-L444: assert it after crash recovery.

As per coding guidelines, “Write behavioral and invariant-based tests.”

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/hermes_cli/test_kanban_completion_audit.py` around lines 355 - 381,
Strengthen the recovery tests by asserting that each recovered task has a
non-null completion_audit_at after enforcement: add this assertion in
tests/hermes_cli/test_kanban_completion_audit.py lines 355-381 for timeout
recovery, lines 384-419 for stuck-worker recovery, and lines 422-444 for crash
recovery. Use the task returned by get_task and preserve the existing status,
run, and claim-lock assertions.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@hermes_cli/kanban_db.py`:
- Around line 8139-8157: Prevent audit requeue while the prior worker is still
alive: in hermes_cli/kanban_db.py lines 8139-8157, 7977-7999, and 8347-8359, use
the existing termination helper, verify liveness after signaling/SIGKILL, and
defer recovery instead of releasing or rearming the claim when the worker
survives.
- Around line 9056-9068: Update the completion-audit handling around
_recover_stale_completion_audit_claim so rate_limited_exit is handled
separately: preserve the rate_limited outcome and backoff, avoid adding the
audit to crashed, and allow the existing quota-blocker handling to run. Keep the
current crashed recovery behavior for non-rate-limited failures.
- Around line 8970-8973: The crash-launch grace query in hermes_cli/kanban_db.py
lines 8970-8973 must join the active current_run_id to task_runs and select
COALESCE(run.started_at, task.started_at) for the timestamp used by the
running-task check. Update tests/hermes_cli/test_kanban_completion_audit.py
lines 429-432 to age the active audit task run rather than the completed parent
task.

---

Nitpick comments:
In `@tests/hermes_cli/test_kanban_completion_audit.py`:
- Around line 355-381: Strengthen the recovery tests by asserting that each
recovered task has a non-null completion_audit_at after enforcement: add this
assertion in tests/hermes_cli/test_kanban_completion_audit.py lines 355-381 for
timeout recovery, lines 384-419 for stuck-worker recovery, and lines 422-444 for
crash recovery. Use the task returned by get_task and preserve the existing
status, run, and claim-lock assertions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1ffef087-4bd8-4df1-8334-2d8cddf7b4d4

📥 Commits

Reviewing files that changed from the base of the PR and between fa86df0 and a41204f.

📒 Files selected for processing (2)
  • hermes_cli/kanban_db.py
  • tests/hermes_cli/test_kanban_completion_audit.py

Comment thread hermes_cli/kanban_db.py
Comment thread hermes_cli/kanban_db.py Outdated
Comment thread hermes_cli/kanban_db.py
@sahilm-ai

Copy link
Copy Markdown
Collaborator Author

auto-review: changes requested.

Matrix checks (U1–U5, C1–C5)

  • U1 in-scope files: PASS.
  • U2 out-of-scope deletions: PASS.
  • U3 no secrets: PASS.
  • U4 AC coverage: FAIL — claimed audit-worker recovery still permits duplicate workers and misclassifies rate limits.
  • U5 mergeable: PASS — CLEAN.
  • C1 CI green: PASS — required GitHub Actions checks passed.
  • C2 type discipline: PASS.
  • C3 lint clean: PASS — CI ruff + ty diff passed.
  • C4 tests touched: PASS, but required recovery regressions are missing.
  • C5 worker identity: PASS — all commits are Sahil (AI).

Code-quality judgment (role-reviewer)

  • [Major] Surviving audit worker is requeued and duplicated: hermes_cli/kanban_db.py:7954-7999, 8125-8157, and 8313-8359 signal a done audit worker then call _recover_stale_completion_audit_claim without checking it actually exited. _terminate_reclaimed_worker / _worker_survived_termination / _defer_reclaim_for_live_worker already implement the required guard in the stale-reclaim path. A forced surviving-PID reproduction returned a requeued audit with cleared claim/current-run and a new completion_audit_at while _pid_alive remained true, permitting the next dispatcher tick to spawn a second worker. Reuse the helper + defer path and add regressions for ignored SIGTERM/SIGKILL in all three controls.
  • [Major] Fresh audits bypass crash launch grace: hermes_cli/kanban_db.py:8969-8988 selects tasks.started_at, but claim_completion_audit_task creates a fresh task_runs.started_at and does not reset the completed parent timestamp. A parent completed longer than the grace window can have its just-spawned audit falsely treated as crashed before PID visibility. Join task_runs through current_run_id, use COALESCE(run.started_at, task.started_at), and age the run—not parent—in the regression.
  • [Major] Rate-limited audits are counted as crashes: hermes_cli/kanban_db.py:9022-9041 recognizes rate_limited, but its audit branch at 9056-9068 always calls recovery with outcome="crashed" and appends to crashed. That skips the normal quota-blocker stamp/backoff at 9093-9103 and can create a quota-wall respawn loop. Preserve rate_limited outcome/error, populate the quota-blocker state, and exclude it from crash accounting.

Findings are mechanical (matrix) or judgment-based (role-reviewer). If a finding looks wrong, leave a counter-comment on the kanban task and Sahil will adjudicate on human-review.

@sahilm-ai

Copy link
Copy Markdown
Collaborator Author

Addressed the remaining CodeRabbit test-coverage nit in 6ce3c56: timeout, stuck-worker, and crash recovery tests now each assert completion_audit_at is restored after requeue. Focused 378-test Kanban matrix passed before the amendment.

@sahilm-ti

Copy link
Copy Markdown
Owner

auto-review: approved, awaiting human merge + kanban_approve.

Matrix checks (U1–U5, C1–C5)

  • U1 in-scope files: PASS — only hermes_cli/kanban_db.py and tests/hermes_cli/test_kanban_completion_audit.py, matching the completion-audit recovery AC.
  • U2 out-of-scope deletions: PASS — no deleted files.
  • U3 no secrets: PASS — diff scan found no credential-shaped additions (the sk- substring was the ordinary word task-run).
  • U4 AC coverage: PASS — restores claim/scheduling/dispatch paths, retains crash-breaker/review paths, and supplies lifecycle regressions.
  • U5 mergeable: PASS — CLEAN at head 6ce3c56b.
  • C1 CI green: PASS — run 29432647829 has all required checks green, including e2e, 8 Python slices, ruff + ty diff, security, attribution, and lock checks.
  • C2 type-discipline: PASS — no new ignore/cast/Any escape hatch; Callable is the only typing addition.
  • C3 lint clean: PASS — required CI lint/type-diff check passed. Local standalone ruff was unavailable in the shared venv; this is not a PR defect.
  • C4 tests touched: PASS — 32 completion-audit tests cover claim/complete/dispatch, stale claims, heartbeat, survivor deferral, timeout/stuck/crash recovery, rate limits, failure rearm, and GC preservation.
  • C5 worker identity: PASS — every PR commit author/committer is Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>.

Code-quality judgment (role-reviewer)

Violations: None. Reviewed the full PR diff and the active-audit lifecycle paths: claims retain done while task_runs are active; recovery routes dead workers through _recover_stale_completion_audit_claim, and live-worker termination failure defers rather than duplicating a worker. All CodeRabbit inline findings are marked addressed at the current head.

Independent verification: scripts/run_tests.sh tests/hermes_cli/test_kanban_completion_audit.py tests/hermes_cli/test_kanban_db.py tests/hermes_cli/test_kanban_dispatcher_crash_breaker.py tests/hermes_cli/test_kanban_human_review.py tests/hermes_cli/test_kanban_merging.py -q → 378 passed, 0 failed.


Findings are mechanical (matrix) or judgment-based (role-reviewer). Sahil retains the substantive merge decision.

@sahilm-ai
sahilm-ai merged this pull request into main Jul 15, 2026
31 checks passed
@sahilm-ai
sahilm-ai deleted the fix/restore-kanban-completion-audit branch July 15, 2026 16:57
sahilm-ti pushed a commit that referenced this pull request Jul 17, 2026
* fix(kanban): restore completion audit dispatch

* fix(kanban): close failed completion audit runs

* test(kanban): keep audit fixtures isolated

* fix(kanban): reclaim stale completion audits

* fix(kanban): recover active completion audits

* fix(kanban): preserve audit worker claims

* test(kanban): assert audit recovery requeues

---------

Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
sahilm-ti pushed a commit that referenced this pull request Jul 21, 2026
* fix(kanban): restore completion audit dispatch

* fix(kanban): close failed completion audit runs

* test(kanban): keep audit fixtures isolated

* fix(kanban): reclaim stale completion audits

* fix(kanban): recover active completion audits

* fix(kanban): preserve audit worker claims

* test(kanban): assert audit recovery requeues

---------

Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
sahilm-ti pushed a commit that referenced this pull request Jul 23, 2026
* fix(kanban): restore completion audit dispatch

* fix(kanban): close failed completion audit runs

* test(kanban): keep audit fixtures isolated

* fix(kanban): reclaim stale completion audits

* fix(kanban): recover active completion audits

* fix(kanban): preserve audit worker claims

* test(kanban): assert audit recovery requeues

---------

Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
sahilm-ti pushed a commit that referenced this pull request Jul 28, 2026
* fix(kanban): restore completion audit dispatch

* fix(kanban): close failed completion audit runs

* test(kanban): keep audit fixtures isolated

* fix(kanban): reclaim stale completion audits

* fix(kanban): recover active completion audits

* fix(kanban): preserve audit worker claims

* test(kanban): assert audit recovery requeues

---------

Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
sahilm-ti pushed a commit that referenced this pull request Aug 24, 2026
* fix(kanban): restore completion audit dispatch

* fix(kanban): close failed completion audit runs

* test(kanban): keep audit fixtures isolated

* fix(kanban): reclaim stale completion audits

* fix(kanban): recover active completion audits

* fix(kanban): preserve audit worker claims

* test(kanban): assert audit recovery requeues

---------

Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
sahilm-ti pushed a commit that referenced this pull request Sep 2, 2026
* fix(kanban): restore completion audit dispatch

* fix(kanban): close failed completion audit runs

* test(kanban): keep audit fixtures isolated

* fix(kanban): reclaim stale completion audits

* fix(kanban): recover active completion audits

* fix(kanban): preserve audit worker claims

* test(kanban): assert audit recovery requeues

---------

Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants