feat(goals): gate continuation on fresh artifact smoke test - #66838
feat(goals): gate continuation on fresh artifact smoke test#66838QuixThe2nd wants to merge 3 commits into
Conversation
Related to merged #50501: this adds an artifact-bound re-verification gate on top of completion contracts rather than duplicating the contract feature. Maintainer direction is needed on the mandatory continuation policy. |
teknium1
left a comment
There was a problem hiding this comment.
Thanks for the thorough artifact-identity and process-cleanup work. The underlying completion gap is real on current main: hermes_cli/goals.py:1428-1481 accepts a structured-goal done verdict without fresh artifact-bound verification.
Problems
- The new applicability path at
hermes_cli/goals.py:2133accepts_latest_goal_verification_status(self.session_id), while the newlatest_verification_status()chooses the latest session-wide workspace rather than evidence associated with this goal. A later structured non-coding or unrelated goal can therefore rerun and be blocked by an earlier coding check. This does not enforce the coding-goal scope documented inwebsite/docs/user-guide/features/goals.md:70. - The new non-coding test covers only
not_applicableevidence; it misses a session with prior unrelated passing evidence.
Suggested changes
- Associate evidence with the active goal/contract or goal generation before enabling this mandatory gate.
- Add the cross-goal regression case above.
This is an automated hermes-sweeper review.
(cherry picked from commit 6b24c6e3fae4eb9a82569a5232e44918cbb4eda4)
38717ba to
375d20d
Compare
|
Addressed in commit |
Description
Structured
/goalcontracts can name objective verification criteria, but the autonomous loop can still advance or finish using evidence from a previous artifact generation.This change adds a mandatory previous-deliverable smoke gate for coding goals with structured verification evidence:
HEAD, tracked contents, hidden tracked modifications, untracked/ignored/explicit paths, and initialized submodule state;PASSis bound to the exact artifact, completion contract/subgoals, and the canonical verification-event payload digest;shell=False, a 30-second timeout, a 1 MiB output bound, artifact-root containment, and rejection of shell/env wrappers, external executables/scripts/tests/configs, and symlink escapes;INCONCLUSIVEremain non-passing;goal_judgereviews bounded evidence for the exact unchanged artifact;PASSis cached; failure or inconclusive review replaces normal continuation ordonewith a repair/re-verification turn;GoalManagerboundary used by CLI, TUI, and gateway callers;The existing SQLite evidence ledger migrates schema v1 to v2 in place. Bare goals and non-coding goals without applicable evidence retain their current behavior.
Type of change
Checklist
How has this been tested?
ruff check .: passedpython scripts/check-windows-footguns.py --all: 772 files scanned, no findingsgit diff --check: passed38717ba6513820d78fb085e28984f4d6c7016f52: PASS with 581 passed in its required matrix0and its detached child was removed;gateway/run.pyandtui_gateway/server.pyThe complete repository suite is left to CI; this PR does not claim a local full-suite pass.
Reviewer focus
Please scrutinize:
done/continuation enforcement;