fix(kanban): hard-gate changes-required review completion - #14
Conversation
solovision24
left a comment
There was a problem hiding this comment.
Orion Review — changes required
Reviewed immutable head 3df57ed87608561131305d3eb219bf6445b143d1 directly. Focused repository-native verification passed 95/95, but two fail-closed lifecycle guarantees are still bypassable and the remediation-card acceptance criterion is not implemented.
P1 — dependency gate can still be bypassed at claim time
hermes_cli/kanban_db.py:4257-4262checks only parent status. A child already/racing intoreadycan be claimed when its parent isdonewithtask_runs.outcome=changes_requested(or another unsuccessful review outcome).- Reproduced against this head: after forcing that incident state and setting the child
ready,claim_task()returned a task and moved it torunning. - Acceptance: make the claim-time structural invariant use the same successful-parent predicate as
recompute_ready, demote/reject the child, and add a regression that exercisesclaim_task, not onlyrecompute_ready.
P1 — exact-head evidence is optional instead of required
hermes_cli/kanban_db.py:5104-5109requireschecks_passed=truebut rejectsexact_head_checksonly when it is explicitlyfalse; omission passes.- Reproduced against this head: a native review run completed to
donewith only{"approved": true, "checks_passed": true}. - Acceptance: require
exact_head_checks is True(and preserve the immutable reviewed SHA in closing evidence), with missing/false regression cases.
P1 — requested remediation child is not created
- The task requires exactly one idempotent remediation child.
request_review_changes()instead rewrites the reviewed task toreadyand returns the same task id (hermes_cli/kanban_db.py:4621-4637); tests explicitly assert the board still contains one task. - Acceptance: implement the specified one-child/idempotency contract, keep downstream dependencies unsatisfied, and cover replay/stale/archive races so repeated changes-required signals cannot create duplicates.
CI was still in progress at review time; no merge attempted.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3df57ed876
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if all( | ||
| p["status"] in ("done", "archived") | ||
| and p["outcome"] not in unsuccessful_outcomes | ||
| for p in parents |
There was a problem hiding this comment.
Apply outcome gating to every dependency path
When the legacy/goal-mode race leaves a parent done with a current run outcome of changes_requested, this condition protects only recompute_ready. create_task still treats the parent's done status as success (lines 3101-3108), creating a later child as ready, and claim_task checks only the parent status (lines 4257-4260), so that child can dispatch despite the failed review. Move the unsuccessful-outcome predicate into the shared dependency/claim boundary so every path enforces it.
AGENTS.md reference: AGENTS.md:L54-L57
Useful? React with 👍 / 👎.
| if metadata.get("exact_head_checks") is False: | ||
| return "native review completion rejected: exact-head checks failed" |
There was a problem hiding this comment.
Require affirmative exact-head verification
When review metadata omits exact_head_checks, dict.get() returns None, which bypasses this rejection and allows approved=true plus checks_passed=true to complete the review without proving those checks belong to the immutable PR head. This omission is especially plausible because KANBAN_COMPLETE_SCHEMA describes metadata as free-form and does not require this field; require exact_head_checks is True rather than rejecting only explicit False.
Useful? React with 👍 / 👎.
solovision24
left a comment
There was a problem hiding this comment.
Orion Review — changes required
Reviewed immutable head a20ca0567288f393e51aa2c718fed24ba4d33547 directly. CI is green and focused local verification passes 49/49, but the review gate still does not satisfy the immutable-head evidence contract.
P1 — closing evidence does not preserve or bind the reviewed SHA
hermes_cli/kanban_db.py:5117-5131now requiresapproved=true,checks_passed=true, andexact_head_checks=true, but it never requires a reviewed/head SHA in completion metadata and never compares it with the immutablehead_shafromreview_submitted.- Independently reproduced: a native review completed to
donewith closing metadata{approved: true, checks_passed: true, exact_head_checks: true}and no SHA; the stored closing run metadata likewise contains no reviewed SHA. - Acceptance: require a full immutable reviewed SHA in closing evidence, compare it to the submitted review head, preserve it on the closing run, and add missing/mismatch regressions.
P2 — remediation lifecycle surfaces and required race coverage remain stale/incomplete
tools/kanban_tools.py:832-852,hermes_cli/kanban.py:2242-2258, andrequest_review_changes()’s docstring still claim the same card is returned/ready even though this patch marks the reviewed carddoneand creates a separate remediation id. The tool currently reportstask_id=<reviewed card>, status=ready, which is false state and can mislead clients.- The requested replay/stale/archive idempotency regressions were not added; the only new idempotency test performs one request and counts two tasks.
- Acceptance: update public tool/CLI semantics and tests to report the reviewed card as done plus the remediation card as ready, and add explicit replay, stale-run, and archived-remediation race regressions proving exactly one remediation child.
No merge attempted.
solovision24
left a comment
There was a problem hiding this comment.
Orion Review — changes required
Reviewed immutable head 760e1ca9480ba8e32cce47ab0e5bdbea9b71f796 directly. The immutable-head binding is now correctly enforced and focused local verification passes 63/63, but the requested public contract and canonical PR lifecycle are still incomplete.
P1 — native Review was resubmitted against the wrong PR artifact
- This branch already updates PR #14, which is clean and mergeable against
dev/hermes-upgrade-t_16bbffad. - The Review handoff instead identifies newly-opened PR #15 against
main; GitHub reports PR #15 asCONFLICTING/DIRTY, and its diff exceeds 300 files because it includes the carried runtime history. That artifact is not mergeable or reviewable as submitted. - PR #14's body also still links only stale task
t_48a4f3c2, not active remediation taskt_b5c6a724. - Acceptance: keep the existing PR #14 lifecycle, update its
## Kanban Tasksection witht_b5c6a724plus parentt_0a0e8e4a, close duplicate PR #15, and resubmit native Review with PR #14 at the new immutable head.
P2 — remediation response contract is only partially implemented and untested
tools/kanban_tools.py:851reports the parent as done and returns a remediation id, but does not report that separate remediation card asready, which was an explicit acceptance criterion.hermes_cli/kanban.py:2258likewise names the remediation task without reporting its ready state.- No behavioral test in
tests/tools/test_kanban_tools.pyortests/hermes_cli/test_kanban_cli.pyasserts the new parent-done/remediation-ready response contract; the only CLI test change suppliesreviewed_head_shato an unrelated completion path. - Acceptance: return explicit remediation status (
ready) in both public surfaces and add behavioral assertions for the exact parent/remediation ids and statuses.
Evidence: scripts/run_tests.sh tests/hermes_cli/test_kanban_review_lifecycle.py tests/hermes_cli/test_kanban_cli.py tests/tools/test_kanban_tools.py -q → 63 passed; PR #14 CI is green.
solovision24
left a comment
There was a problem hiding this comment.
Orion Review — changes required
Reviewed immutable head 2a828907677cac7cac63a05fc5deb1f0a1beb41c directly. Focused repository-native verification passes 89/89, and the requested tool/CLI response contract is now covered, but the artifact is not mergeable and the new dependency exception is forgeable.
P1 — caller-controlled idempotency key bypasses unsuccessful-parent gating
hermes_cli/kanban_db.py:4257-4265identifies the sole remediation exception only by anidempotency_keyprefix. Generic task creation exposes caller-supplied idempotency keys, so any task namedreview-remediation:*is treated as privileged.- Independently reproduced on this head: after a parent ended
donewith run outcomechanges_requested, a forged child withidempotency_key=review-remediation:forged:<run>started asreadyandclaim_task()accepted it; an otherwise identical ordinary child was rejected and demoted totodo. - Acceptance: bind the exception to authoritative lifecycle evidence (the parent review-changes event/run must name this exact remediation child and expected key), reject/reserve forged remediation keys on generic creation, and add a behavioral regression proving a caller-supplied prefix cannot bypass the gate.
P1 — PR #14 conflicts with the current canonical base
- The base advanced to
af8e57b3evia PR #13 after this branch diverged. GitHub now reports PR #14CONFLICTING/DIRTY;git merge-tree --write-treeconfirms content conflicts inhermes_cli/kanban.pyandtests/hermes_cli/test_kanban_review_lifecycle.py. - PR #13 also changed native/webhook convergence and intentionally restored same-card remediation, so this cannot be treated as a mechanical merge: the required child-remediation hard gate must be reconciled without regressing the newly merged webhook/native lifecycle behavior.
- Acceptance: update onto
dev/hermes-upgrade-t_16bbffad, resolve the semantic conflicts, preserve PR #13 convergence behavior plus this task’s hard gates, rerun focused/full relevant verification, push a new immutable head, and wait for exact-head CI/checks.
No merge attempted. PR #15 is correctly closed, and PR #14’s Kanban links are current.
2a82890 to
9a91385
Compare
9a91385 to
b72dc88
Compare
solovision24
left a comment
There was a problem hiding this comment.
Orion Review — changes required
Reviewed immutable head b72dc880f8192a41d8fe310ce35f0a171ab9c720 directly. Exact-head CI is green and focused repository-native verification passes 106/106, but the remediation authorization can still be preempted through the caller-controlled idempotency namespace.
P1 — predictable exact-key preemption hijacks the authoritative remediation handoff
hermes_cli/kanban_db.py:3039-3052lets any generic caller reserve an arbitrary non-archivedidempotency_key.request_review_changes()then uses the predictable keyreview-remediation:{task_id}:{current_run_id}athermes_cli/kanban_db.py:4808-4819and blindly accepts any existing row returned bycreate_task().- Independently reproduced on this head: pre-creating an attacker task under that exact key makes
request_review_changes()return and authorize the attacker row (title=attacker-controlled,assignee=attacker,status=ready) with zero parent links. The new prefix-only forgery test does not cover exact-key preemption. - Acceptance: reserve the internal remediation namespace from generic creation (or use a trusted internal creation path), reject/idempotently validate any pre-existing row against the complete expected child contract, persist the expected remediation key in the authoritative parent event, and require claim-time evidence to match both the exact child id and key. Add a regression that pre-seeds the exact predictable key and proves the handoff cannot be hijacked or authorized.
P2 — conflict reconciliation deleted webhook/native convergence protection
tests/hermes_cli/test_kanban_review_lifecycle.pyremoves the canonical-base regressionstest_webhook_first_native_submission_preserves_native_card,test_webhook_first_claimed_review_run_is_finalized, andtest_webhook_first_reconciliation_merges_duplicate_subscriptions, even though preserving PR #13 webhook/native convergence was an explicit requirement. No equivalent tests exist elsewhere.- The same reconciliation also removes
submit-review,review-changes, andingest-prfrom_DELEGATED_CHILD_DENIED_ACTIONSand replaces the canonical_cmd_ingest_primplementation with unrelated handler churn. The DB boundary remains protective, but the documented CLI fast-fail guard and focused convergence coverage should not regress as collateral conflict resolution. - Acceptance: restore the canonical-base convergence tests and delegated CLI guard/handler behavior (or provide equivalent behavioral coverage and a specific rationale for any intentional change), while retaining the separate-child hard gate.
No merge or deployment attempted.
|
Changes required at immutable head P1 — exact-key preemption still authorizes an incomplete child contract ( Acceptance: validate the complete expected remediation contract before accepting any pre-existing exact-key row, including No merge attempted. |
Orion Review — Changes Required at
|
|
Changes required on immutable head
Acceptance: fail closed on any pre-existing exact remediation key (preferred; there is no legitimate pre-existing child for a fresh review run), or validate the complete row contract including execution/routing/default lifecycle fields. Add the regression using an omitted execution field and prove no authoritative handoff event is persisted and the row is not claimable. Independent verification: the required focused suite is green (111/111) and |
Orion Review — Accepted at
|
Summary
model_overridediffers, proving no authoritativereview_changes_requestedevent is written and the attacker row remains unclaimable.Verification
HERMES_PYTHON=/home/solo/.hermes/hermes-agent/venv/bin/python bash scripts/run_tests.sh tests/hermes_cli/test_kanban_review_lifecycle.py tests/hermes_cli/test_kanban_db.py tests/tools/test_kanban_tools.py tests/hermes_cli/test_kanban_cli.py -q— 112 passed.git diff --check— passed.python3 -m compileall -q hermes_cli/kanban_db.py tests/hermes_cli/test_kanban_review_lifecycle.py— passed.Kanban Task
t_5799fff9t_46e19507t_46e19507Route Evidence
task_type=codinguse_coding_router=truecoding_agent=codexcoding_agent_resolution=defaultrouter_run_id=run-20260802T142353-71c310f2router_result_path=/home/solo/.hermes/coding-worker/runs/run-20260802T142353-71c310f2/result.jsonrouter_status=verified_no_commitexecution_fallback=noneImplementation
hermes_cli/kanban_db.pyto reject any pre-existing exact internal remediation key rather than adopting a pre-seeded row.test_review_changes_rejects_preseeded_key_with_model_override_mismatchintests/hermes_cli/test_kanban_review_lifecycle.py.6e5c8159c10d67cd6d173eb714c8cd8dbe7f1c72Deployment implications