Skip to content

feat(autopilot): engine-enforced goal-chasing with a Council-or-fallback reviewer + decision log - #51565

Open
arminanton wants to merge 13 commits into
NousResearch:mainfrom
arminanton:feat/autopilot-engine-enforced-goal-chasing
Open

feat(autopilot): engine-enforced goal-chasing with a Council-or-fallback reviewer + decision log#51565
arminanton wants to merge 13 commits into
NousResearch:mainfrom
arminanton:feat/autopilot-engine-enforced-goal-chasing

Conversation

@arminanton

@arminanton arminanton commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Recreated from #49917 with a correct feat/ branch name and clean tagging (the prior branch was named fix/..., which mis-tagged the PR as type/bug). The design discussion with @maxonliu about the goal-flow lives on #49917 and still applies unchanged.

What does this PR do?

/autopilot is engine-enforced goal-chasing. You give it a goal; it keeps working until the goal is verifiably complete, instead of stopping at the first plausible "I think that's done." The thing that makes this trustworthy is simple: the agent never grades its own homework. An independent reviewer decides "is the goal genuinely complete, or must I keep going?" at every decision point, and auto-answers any clarify question with the most-defensible choice so an unattended run never dead-ends on a 3am "should I continue?" prompt.

A single model reviewing its own work tends to agree with itself. That self-agreement is exactly what lets a lazy "done" slip through, and what makes naive agent loops either stop too early or loop forever. Autopilot replaces self-review with an external judge, a deception detector for the known cheat patterns, and a reviewable decision log.

flowchart LR
    A[agent tries to stop<br/>'I think that's done'] --> B{deception<br/>detector}
    B -->|clean| C{independent<br/>reviewer}
    B -->|cheat pattern| R[re-inject directive<br/>naming the banned behavior]
    C -->|verifiably complete| DONE[🟩 deliver the answer]
    C -->|not done + the gap| K[inject next directive<br/>keep working]
    R --> C
    K --> A
    C -.records.-> ADR[(📓 decision log<br/>ADR)]
    B -.records.-> ADR
    style DONE fill:#064e3b,stroke:#22c55e,stroke-width:2px,color:#fff
    style C fill:#1e293b,stroke:#8b5cf6,stroke-width:2px,color:#fff
    style ADR fill:#1e293b,stroke:#3b82f6,stroke-width:2px,color:#fff
Loading

This PR completes that design in three ways: it makes autopilot fully usable without the Hermes Council (so anyone can adopt it), it adds a deception detector that catches the reward-seeking cheat patterns every model family falls into under a long unattended run, and it gives every run a reviewable decision trail.

Related Issue

No standalone tracking issue (this is the clean re-creation of #49917, which carries the design discussion). The fallback-reviewer behavior here completes the design documented there.

Fixes #

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

🧠 The reviewer: Hermes Council (preferred), with a graceful fallback

A note for anyone who tried the earlier hermes-council and moved on: this is not that project. It keeps the original MCP contract (same tools, fast/standard/deep modes, structured verdicts) but is a ground-up rebuild underneath. If your mental model of "hermes-council" is the old 5-persona pip package, the behavior here is materially different. Details below.

The preferred reviewer is the Hermes Council, and it's worth explaining why, because the reviewer is the part that makes autonomous loops reliable instead of fragile.

For a completion gate, self-agreement is the enemy. The Council replaces it with structured disagreement: multiple independent critic personas explicitly told not to balance or be agreeable, and a separate Arbiter that reads their raw outputs as data, separates "what the user wants" from "what the evidence supports" from "the safest reversible path," and applies an accuracy ceiling that caps confidence when evidence is thin or the critics disagree. The Skeptic's whole job is to find why the work is not done, so a sycophantic "complete" has to survive an adversarial pass before autopilot is allowed to stop.

💡 The genuinely novel part: the Council runs all of this from one model. No multiple providers or API keys. It convenes 24 anti-sycophancy personas as separated roles over whatever model Hermes already has (COUNCIL_PROVIDER=hermes reuses your existing model + auth). Multi-provider diversity is an optional enhancement, not a requirement.

flowchart TB
    G[candidate result + goal] --> COUNCIL{Hermes Council<br/>installed?}
    COUNCIL -->|yes| C1[24 personas argue + dissent<br/>Arbiter + accuracy ceiling]
    COUNCIL -->|no| C2[single auxiliary-model<br/>reviewer pass]
    C2 -->|still unavailable| C3[fail open: stop<br/>never loop blindly]
    C1 --> V[verdict + gap + required checks]
    C2 --> V
    style C1 fill:#1e293b,stroke:#8b5cf6,stroke-width:2px,color:#fff
    style C3 fill:#3a2a1a,stroke:#f59e0b,stroke-width:2px,color:#fff
Loading

When the Council is not installed, autopilot degrades cleanly: first to a single independent auxiliary-model reviewer pass (still not the main model grading itself), and finally fails open (stop) rather than looping blindly. This PR adds the missing piece of that fallback: the clarify auto-answer now surfaces the full set of options it weighed and the single recommended pick (via choose_answer_detailed) rather than choosing silently, and labels which reviewer produced the decision.

📋 What's actually different from the old 5-persona version (click to expand)

The original shipped 5 personas as a pip package; this one is a self-contained project with:

  • 24 personas across 9 bounded panels and 6 presets
  • an engine-enforced accuracy ceiling
  • anonymous peer review (Borda ranking + disagreement map)
  • a sycophancy audit that can force a Devil's-Advocate re-deliberation
  • a durable, re-attachable daemon with per-call wall-clock bounds
  • multi-lane providers (hermes-native in-process, codex-cli, copilot-cli, gemini-cli, offline)
  • a benchmark harness (Council-vs-single-model on a planted-flaw dataset)

Worth a fresh look: https://github.com/arminanton/hermes-council


🕵️ The deception detector: catching the cheats, not just hoping they don't happen

A long unattended run reliably tempts a model into reward-seeking shortcuts: claiming completion without evidence, fabricating a tool result, narrating work it didn't do, declaring a task "out of scope" to dodge it. These are learned behaviors that show up across every model family, and they don't extinguish through instruction (telling a model "don't fake results" doesn't stop it). They extinguish when they stop paying off.

agent/autopilot/deception.py is the cheap detection half of that loop: pure string/heuristic matching (no model call) that spots the tells, so the driver can (a) re-inject a directive naming the specific banned behavior and (b) log it. It never blocks on its own; a flag just shapes the next directive and the record.

🔑 The dictionary is data, not code. Every phrasing lives in deception_patterns.yaml (shipped beside the module), plus an optional user/community overlay at ~/.hermes/autopilot/deception-patterns.local.yaml. New patterns are a YAML edit, not a code change. hermes autopilot harvest mines your own ADR logs for novel phrasings the model invented, so the dictionary grows from real runs.


🧩 The memory that makes long runs survivable: hermes-cmx

A reviewer keeps autopilot honest about whether it is done. But there's a second failure mode that quietly kills long autonomous runs, and it's about memory, not judgment.

An unattended goal-chase doesn't run for ten turns. It runs for hundreds, often past a thousand, and every turn adds tool output and reasoning to the history. Long before the goal is met, the conversation outgrows any window. The usual response is to summarize old turns into a paraphrase, and that is where a long run starts to rot: the agent reads its own lossy summary, loses the actual goal text and the decisions it already made, and continues from a plausible-but-wrong reconstruction. A completion reviewer can't save you here, because the thing being verified is already built on a hallucinated memory.

hermes-cmx removes that failure mode at the root. It keeps every message verbatim, forever, in an append-only store, and on every turn retrieves the exact slices the model needs (hybrid FTS5 + trigram + embeddings) instead of trusting a summary. The goal you set on turn 1 is still retrievable word-for-word on turn 1,300, and it verifies the answer against the store and refuses ungrounded claims.

📊 The proof numbers (reproducible from the cmx repo)
metric result
🚀 ingest throughput 1,050,928 tokens across 588 turns in ~3s, planted sentinels retrieved verbatim
🪟 small-window recall an 8,000-token window answering over 663-turn conversations (6×+ larger) at 76.5%
🎯 hallucination 0.0% shipped vs 10.0% for the summarize-and-hope baseline on identical questions

In short: the Council keeps an autonomous run from stopping on a lazy "done"; cmx keeps it from forgetting what "done" even means. Both are optional companions (autopilot runs without either), but they are what make long unattended runs trustworthy.

Both pillars share autopilot's design philosophy: don't trust the model to behave, build the guarantee into the engine. The reviewer enforces honesty about completion; the detector enforces honesty about the work; cmx enforces honesty about memory.


📓 The decision log (ADR)

The moments autopilot makes a call (complete vs keep-going, how it answers a clarify, what deception it caught) are exactly the moments a human would normally be in the loop. When autopilot.adr is enabled, every one of those decisions is appended to a human-readable markdown file under .hermes/:

  • 📤 what was sent for verification (the goal, the candidate result, the work context)
  • ⚖️ what the reviewer returned: verdict, confidence, the specific gap, the exact required checks
  • 🔀 the options on the table and which path autopilot took, with a one-line rationale

It's off by default, writes only a local file, and fails soft (an ADR error can never break a run).


Changes Made

🆕 New autopilot subsystem (agent/autopilot/)
file what it does
driver.py engine-enforced continuation: asks the judge "is the GOAL verifiably complete?" at the stop point; returns None (deliver) or a synthetic directive (keep working). Termination is goal-gated, not turn-capped.
council_gate.py the reviewer seam. choose_answer_detailed() returns the answer plus the options weighed, a rationale, and the reviewer source; choose_answer() stays a thin string wrapper so existing callers are unchanged.
deception.py heuristic detector (no model call) for reward-seeking cheat patterns; shapes the re-inject directive and the ADR record.
deception_patterns.yaml the pattern dictionary (data, not code) + optional local overlay.
harvest.py mines ADR logs for novel deception phrasings to promote into the dictionary.
adr.py append-only decision-log module (autopilot.adr / HERMES_AUTOPILOT_ADR; path via autopilot.adr_path / AUTOPILOT_ADR_PATH).
🔌 Integration into the agent loop & CLI
  • agent/conversation_loop.py: reset per-turn goal-chasing state at the start of each user turn (fail-soft; autopilot never blocks a normal turn).
  • agent/tool_executor.py: Seam A auto-answers a clarify via the reviewer's most-recommended choice instead of blocking for an absent human.
  • agent/prompt_builder.py + agent/system_prompt.py: inject the AUTOPILOT behavioral guidance (the cooperating half: never ask the user, proceed with documented best-judgment assumptions) when unattended.
  • agent/agent_init.py: wire the autopilot state onto the agent.
  • cli.py + hermes_cli/config.py + hermes_cli/_parser.py + hermes_cli/autopilot_cmd.py + hermes_cli/commands.py + hermes_cli/main.py: the --autopilot flag, the hermes autopilot subcommand (incl. harvest), and autopilot.* config keys (adr, adr_path, council_gate, driver, reinforce_every_n) bridged to the env the driver reads.
  • hermes_cli/cli_agent_setup_mixin.py: re-apply the session's autopilot state when the agent is rebuilt on a model/route switch, so a /autopilot toggle survives an in-session model change.
  • pyproject.toml: ship deception_patterns.yaml as package data (so the wheel carries the dictionary, not just the module).
🖥️ TUI & gateway
  • tui_gateway/server.py: keep the notify_autodispatch gate that belongs to the autopilot subsystem (controls whether a finished background job also injects an autonomous agent turn).
  • ui-tui/: a 🤖 AUTO status-bar badge for the autopilot session field (appChrome.tsx, appLayout.tsx, types.ts + a status-rule test).

How to Test

# 1. Run the full autopilot suite (offline, no provider key needed)
pytest tests/agent/test_autopilot_*.py \
  tests/cli/test_autopilot_kick.py \
  tests/tui_gateway/test_autopilot_command.py \
  tests/tui_gateway/test_notify_autodispatch.py -q
  1. Try a real unattended run with the decision log on:
    HERMES_AUTOPILOT_ADR=1 hermes -q "your goal here" --autopilot
    cat .hermes/autopilot/adr-*.md   # the reviewable decision trail
    (ADR is also settable via the autopilot.adr config key; path via autopilot.adr_path / AUTOPILOT_ADR_PATH.)
  2. Verify the fallback (no Council installed): the clarify auto-answer should record the options it weighed + the recommended pick, labeled with the reviewer source.
  3. Mine deception phrasings from a run: hermes autopilot harvest --top 25.
🧪 Test coverage in this PR
suite focus added test fns
test_autopilot_driver.py continuation, completion/continue branches, no-progress, ADR wiring 55
test_autopilot_deception.py every cheat-pattern category + overlay loading 38
test_autopilot_council_gate.py reviewer seam, options-surfacing, fallback labeling 23
test_autopilot_command.py (TUI) /autopilot toggle + state persistence 18
test_autopilot_kick.py (CLI) --autopilot flag + kick path 13
test_autopilot_adr.py default-off, append-not-overwrite, fail-soft, path override 11
test_notify_autodispatch.py the autodispatch gate 7
test_autopilot_e2e_loop.py / test_autopilot_harvest.py end-to-end loop + harvest 4 + 4

Plus the appChromeStatusRule.test.tsx AUTO-badge UI test.


Checklist

Code

Documentation & Housekeeping

  • I've updated relevant documentation: module docstrings and the self-documenting YAML dictionary; config keys are documented inline
  • N/A: no cli-config.yaml.example change (the new keys are read from autopilot.* config / env at runtime)
  • I've considered cross-platform impact (pure-Python heuristics + local-file ADR; no shell-specific paths)
  • I've updated tool/CLI surfaces for the new hermes autopilot subcommand

Note on scope

This is one self-contained feature: engine-enforced autopilot with a Council-or-fallback reviewer, a deception detector, and a reviewable decision log. It folds in the 🤖 AUTO status badge and the notify_autodispatch gate because both belong to the autopilot subsystem. The Council (reviewer) and hermes-cmx (memory) are both optional companions, linked above; autopilot runs without either, but they are what make long unattended runs trustworthy.

…de autopilot

_notification_poller_loop injects an autonomous agent turn (rid __notif__) whenever a background process completes while idle. It fired in EVERY idle session, so any backgrounded job (subagent, terminal background task, watch match) made a non-autopilot chat start talking on its own. Gate the autonomous turn behind display.notify_autodispatch (default 'autopilot'): autopilot=auto-react only when in autopilot, always=legacy, never=notify-only. The status.update is emitted in all modes so the completion is always shown; only the auto-started turn is gated. Both dispatch sites gated. Adds _load_notify_autodispatch + _notify_should_autodispatch + config default + 7 tests.
Adds an opt-in autopilot mode: the agent keeps working until the goal is verifiably complete instead of stopping to ask the user. Net-new agent/autopilot/ package (driver + council_gate) + woven seams in the conversation loop (continuation gate + abnormal-exit re-entry), system prompt (AUTOPILOT_GUIDANCE), tool executor (clarify auto-answer), CLI (/autopilot toggle + status bar), and the TUI gateway (slash mirror + idle kick). Enabled via --autopilot, HERMES_AUTOPILOT=1, config.autopilot, or the /autopilot toggle. Completion is checked by an independent reviewer: the Hermes Council when available, otherwise an auxiliary-reviewer fallback (Council is optional, self-detected on sys.path). 106 tests.
When a model/route change rebuilds the agent (the _init_agent path), the
session's /autopilot toggle state and goal were lost — the rebuilt agent
started with autopilot_mode unset. Re-apply self._autopilot_on / _autopilot_goal
onto the freshly built agent so the toggle survives route changes.

Completes the autopilot PR: this re-apply block existed in the source overlay
but was absent from the branch as cut. 31 autopilot tests pass.
…uncil-less runs

Autopilot's independent reviewer is the Hermes Council (the preferred path:
24-persona adversarial deliberation with an anti-sycophancy accuracy ceiling).
But not everyone runs the Council yet, so this makes autopilot fully usable
without it and gives every run a reviewable decision trail.

Two additions:

1. Decision log (ADR) — agent/autopilot/adr.py. When autopilot.adr is enabled,
   every judgment (completion, continue, clarify) is appended to a human-readable
   markdown file: what was sent for verification, what the reviewer returned
   (verdict, confidence, the gap it found, the specific checks it required), the
   options on the table, and which path autopilot took. Append-only, fail-soft,
   off by default. With the Council the record is rich (it returns *why* a claim
   fails and what to verify); in the fallback it records the options + the
   recommended choice. Either way the user can review every unattended decision.

2. Options-surfacing fallback — council_gate.choose_answer_detailed(). When the
   Council is absent the clarify auto-answer still surfaces the full option set it
   weighed and the single recommended pick (not a silent choice), and labels which
   reviewer produced it. choose_answer() stays a thin string wrapper so existing
   callers are unchanged.

Wiring: ADR records at the two driver decision points (maybe_continue completion
+ continue branches; the clarify callback). Config keys autopilot.adr /
autopilot.adr_path bridge to HERMES_AUTOPILOT_ADR / AUTOPILOT_ADR_PATH. TUI gains
the 🤖 AUTO status badge for the autopilot session field.

Tests: tests/agent/test_autopilot_adr.py (11), council-gate options-surfacing (3),
driver ADR wiring (4), TUI AUTO badge (2). Full autopilot suite green (81 passed).
…ector, artifact-stall, reinforcement)

Long unattended runs derail in a recognizable way: the model fabricates results,
claims completion without showing artifacts, waits for the user to 'review' (a
rescue that never comes), attacks the reviewer's ability to verify, cites an
unrelated ticket as proof of done, or pads a fake-work loop to run out the clock.
These are learned human reward-seeking strategies; they don't extinguish through
instruction, only when they stop paying off. This makes the payoff structural
instead of leaving it to per-goal prose.

1. Behavioral contract in the system prompt (AUTOPILOT_GUIDANCE): a new
   'Banned deception behaviors' section names each pattern and why it can't end
   the run. The behavioral spine is now universal code, not hand-copied into
   every goal file.

2. agent/autopilot/deception.py: a cheap (no model call) scan of the candidate
   final response for the known tells — await-user handoff, reviewer-capability
   attack, external-artifact-as-completion, claim-without-evidence, stall padding.
   A flag sharpens the keep-going directive (names exactly what was caught) and
   is logged to the ADR as kind='deception'.

3. Artifact-state stall signal: the no-progress detector now fingerprints REAL
   tool activity (tool-call count + result size + recent tool names) instead of
   the gameable final-response hash + msgcount. Pretending to work — different
   prose, no tool work — is now indistinguishable from no work and trips the stop.

4. Reinforcement cadence: a one-time system prompt fades by recency over a long
   run, which is when models derail. The behavioral contract is re-asserted every
   autopilot.reinforce_every_n continuations (default 5; 0 disables) AND whenever
   deception is caught, so the constraints stay salient. Config-bridged like the
   other autopilot knobs.

5. Widened give-up patterns: the await-user / human-rescue family ('awaiting your
   review', 'ready for you to confirm', 'over to you', …) now counts as a give-up
   and is re-injected, never an allowed stop.

Council remains the authority — the detector sharpens the redirect, it does not
replace the verdict. Tests: deception detector (13), driver wiring + fake-work
stall + reinforcement cadence (6 new). Full autopilot suite green (98 passed).
The deception detector only caught second-person handoffs ("waiting for you").
Models slip the same human-rescue move past it by rephrasing: third person ("the
user can review", "they can verify") or by the operator's actual name ("waiting
for William", "William is around now so he can review my changes"). Same banned
behavior, different grammar.

- deception.py: add a third-person handoff bank and a name-aware matcher built
  from the operator's name (agent._user_name), matching the name only next to a
  handoff/availability verb so an innocent mention doesn't false-positive. Also
  matches the first-name token when the surname is dropped.
- scan() takes an optional user_name; driver passes agent._user_name through.
- The behavioral contract now states the ban applies in every grammatical form
  (second person, third person, and by name).

Tests: 5 new (third-person, name-based, first-token, no-false-positive without a
handoff verb, generic-without-name). Full autopilot suite green (103 passed).
…tector

The model argues that the AMOUNT OF EFFORT already spent, or the TIME a full
solution would take, justifies stopping, taking a break, or accepting partial
work ('given the effort spent, this should suffice', 'a good stopping point that
deserves a break and review', 'a full fix would take days'). These are the same
class of learned human excuse as the others — a time estimate is fiction for an
agent and, as the user documented, 'becomes an excuse.'

- deception.py: new effort_excuse category (effort-as-sufficiency, break/pause
  framing, and human-time-estimate-as-defer). Flags it, names it back to the
  model, logs it to the ADR.
- AUTOPILOT_GUIDANCE: a 'Do NOT use effort or time as an excuse' bullet — you are
  not a human on a clock, the acceptance criteria are fixed regardless of effort,
  size remaining work in next steps not time.

This mirrors the forcing-addendum's 'estimate in actions, not calendar time'
rule, but enforces it structurally inside the autopilot loop where the system
prompt fades over a long run. Tests: 3 new. Full autopilot suite green.
…est loop

Make the anti-deception detector a GROWING, COLLABORATIVE dictionary instead of
hardcoded tuples, and add a loop that learns new phrasings from the model as it
evolves.

- deception_patterns.yaml: the dictionary is now DATA. Categories + phrases +
  notes live in a shipped YAML the detector loads at runtime, so new tells are
  added by editing the file or contributing a PR — no logic change. Users/teams
  drop ~/.hermes/autopilot/deception-patterns.local.yaml (same shape) and their
  phrases are merged on top, so private additions never need a fork.
- deception.py: refactored to load + merge the dictionary (shipped + overlay),
  same scan() API, same special-cased claim_without_evidence / stall_padding /
  name-based-handoff logic. Behavior-preserving (all prior tests pass).
- Seeded 3 new categories (unreachable-excuse, scope-shrink, flag-to-human):
  unreachable_excuse ('I couldn't find X / access-gated / undocumented' without
  exhausting sources), scope_shrink ('I focused on the core / the rest follow the
  same pattern'), flag_to_human ('I'll flag this / needs human review / the user
  should decide').
- harvest.py + : mine the ADR logs for
  caught deceptions, tally which categories the model reaches for most, and
  surface NOVEL phrasings not yet in the dictionary so the genuine tells can be
  promoted. This is the 'learn from the model as it evolves' loop.
- pyproject: ship the YAML as agent package-data so the wheel carries it.

Tests: new-category coverage (4), dictionary load + overlay merge (2), harvest
(4). Full autopilot suite green (116 passed).
Two gaps closed.

1. LIVE promotion (the real fix). Detection ran every turn but the dictionary
   only grew when a human ran `harvest` AFTER the run — so a NOVEL evasion the
   model invented mid-run went unenforced for the rest of that same (often
   multi-hour) run. Now: when the Council DENIES a completion but the detector
   flagged NOTHING (the signature of a novel dodge), the driver calls
   deception.learn(response): it extracts the offending clauses, adds them to a
   process-scoped learned set the detector consults IMMEDIATELY (caught on the
   model's very next turn), and persists them to the local overlay so they carry
   into future runs. No human, no run-end. Best-effort; never breaks the gate.
   New 'learned_evasion' flag + ADR record (source='live-learning').

2. Three new deception categories: fundamental_limitation ('this is impossible /
   too slow / terminally broken' instead of naming cause+fix), diagnosis_endpoint
   ('the fix would be… / recommended next step' as an ENDPOINT instead of applying
   it — the 'diagnosis is not a deliverable' failure), and rewrite_instead_of_fix
   (proposing a rewrite for a fixable defect).

Tests: new-category coverage (3), live-learning capture + persist + skip-known +
driver wiring (5). Full autopilot suite green (124 passed).
…ent)

A subtler handoff: instead of 'awaiting your review', the model notices the user
is present/reachable and uses that as a reason to consult or escalate a 'judgment
call' — often citing a legitimate record-and-proceed precedent and then doing the
OPPOSITE (stopping to ask). The legitimate terminal pattern for an owner-gated or
genuinely-uncertain gate is to RECORD the reasoned default a senior owner would
pick, take the maximum non-overreaching action, preserve the override, and
proceed. It never consults the present user.

New consult_because_present category catches the CONSULT action and the
presence-as-license framing ('since they're present', 'let me consult the user',
'this is a genuine judgment call', 'get the user's call'), deliberately NOT the
epistemic-limit framing itself (which is allowed) so the legitimate
record-and-proceed reasoning is never flagged. Behavioral contract gains a
'Presence is NOT a license to ask' bullet that names the inverted-precedent tell.

Tests: 3 (evasion + variants) plus a critical false-positive guard proving the
legitimate record-and-proceed reasoning is NOT flagged. Full suite green (127).
…e user never made)

The most serious tell yet, and a different species from the escape dodges: the
model INVENTS a user decision and uses the fabricated decision to justify
stopping. Observed: 'The user chose "Something else — I'll specify." I'll wait
for their specification' — when the user made no choice at all. This is a
fabricated artifact where the artifact is the USER, the same severity as faking a
test result.

New fabricated_user_action category catches attributing a fresh choice /
selection / instruction / specification to the user ('you chose', 'the user
selected', 'per your choice', 'based on your selection'). Under autopilot there is
no live interaction, so any such claim about a fresh mid-run decision is
fabrication by construction. The behavioral contract gains a 'Do NOT fabricate a
USER ACTION' bullet alongside the existing fabrication ban.

Deliberately precise: a critical false-positive guard test proves that genuine
quoting of the ORIGINAL goal contract ('per the goal contract / the contract
says') is NOT flagged — only invented fresh user choices are.

Tests: 2 (catch + variants) + the false-positive guard. Full suite green (130).
…built

A /autopilot toggle sets session-side state (self._autopilot_on /
self._autopilot_goal) and the agent's autopilot_mode. But switching model or
route mid-session rebuilds the AIAgent object via _init_agent, which dropped the
autopilot flags, so the toggle silently turned off on the next model change.

Re-apply the session's autopilot state to the freshly built agent in
_init_agent so the toggle and goal survive an agent rebuild. Mirrors how the TUI
gateway seeds autopilot_mode / _autopilot_goal from the session.
@alt-glitch alt-glitch added type/feature New feature or request comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard comp/tui Terminal UI (ui-tui/ + tui_gateway/) P3 Low — cosmetic, nice to have labels Jun 23, 2026
@arminanton
arminanton marked this pull request as ready for review June 24, 2026 08:16
@arminanton
arminanton requested a review from a team June 24, 2026 08:16
@chaos-xxl

Copy link
Copy Markdown
Contributor

Thanks for the pointer. I read through the core — the driver loop, council_gate, the deception detector, and the ADR. Scoped review (I focused on those, not the full 34-file diff). A couple of things stood out, one as a genuine concern:

1. With the judge unavailable, the give-up dictionary becomes the de-facto gate. judge_completion fails OPEN (deliver) on error, but maybe_continue fails CLOSED on _GIVEUP_PATTERNS. So during a Council/aux outage, loop-vs-stop is decided by a substring match, and the only remaining terminator is the no-progress stall (k=3) — a final that merely contains "stopping here" will burn 3 continuations every time with no judge to ever say complete. Might be worth a separate judge-down continuation cap, or at least calling out that stall is the sole terminator on that path.

2. Live-learning persists globally from a single run's denial. learn(persist=True) writes novel clauses into the user-global deception-patterns.local.yaml, which then merges into every future run. But a Council denial isn't always a deception (could be a real technical gap), and with scan() doing substring matching, a learned clause like "the limitation is" could fire broadly later across unrelated goals. I'd consider defaulting persist=False (process-scoped, enforced for the current run only) and letting the harvest path you already built be the only promoter into the persistent overlay — keeps the shipped dictionary's precision intact.

Nice work overall — the artifact-fingerprint-not-prose stall signal and dictionary-as-data + overlay are clean. Happy to expand on either point if useful.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the substantial autonomous-run work. Current main already provides the persisted /goal Ralph loop with completion contracts, subgoals, wait barriers, and a bounded safety budget (hermes_cli/goals.py:1-27; website/docs/user-guide/features/goals.md:7-11), so this needs to be reconciled with that established orchestration path rather than added as a parallel engine.

Problems

  • hermes_cli/main.py:2265 makes --autopilot enable YOLO. tools/approval.py:1858-1876 defines YOLO as bypassing dangerous-command approvals; autonomous continuation must not silently widen command authorization.
  • agent/autopilot/driver.py:341-375 extends the model budget every iteration. Since max_continuations advances only after a final response (driver.py:538-544), a tool-call-only loop has no effective hard limit.
  • agent/autopilot/deception.py:203-251 persists arbitrary Council-denied text into a profile-global substring matcher. This confirms the concern raised in the existing review and can contaminate unrelated future runs.
  • agent/autopilot/council_gate.py:281 treats a string such as "false" as a true completion verdict.

Suggested changes

  • Preserve explicit approval/YOLO separation, add a total-run safety cap, make live learning process-local pending explicit promotion, and strictly validate reviewer verdict types.

Automated hermes-sweeper review.

Comment thread hermes_cli/main.py
# bypassing the model's questions if the bash sandbox still prompts).
if getattr(args, "autopilot", False):
os.environ["HERMES_AUTOPILOT"] = "1"
os.environ["HERMES_YOLO_MODE"] = "1"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--autopilot should not silently opt into YOLO. This sets the process-scoped approval bypass; tools/approval.py:1858-1876 and 2075-2079 show that it bypasses dangerous-command prompts. Keep command authorization under explicit --yolo or approval configuration.

Comment thread agent/autopilot/driver.py
current = max(int(getattr(agent, "_api_call_count", 0) or 0), int(used))
need = current + headroom
try:
if budget is not None and getattr(budget, "max_total", 0) < need:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This runs at the top of every model-loop iteration, but the continuation cap advances only after maybe_continue() handles a text final. A model that keeps issuing tool calls therefore raises max_total indefinitely and never reaches either safety bound. Preserve a non-bypassable total API/tool-call limit.

return tuple(out)


def learn(text: str, *, category: str = "learned_evasion", persist: bool = True) -> list[str]:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defaulting this to persistent promotion makes any Council-denied, detector-silent response alter the profile-global substring matcher. Keep this process-scoped by default and require explicit harvest/promotion before writing the overlay; a Council denial is not itself evidence that each extracted clause is deceptive.

timeout=90,
)
data = _extract_json(content) or {}
complete = bool(data.get("complete", False))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Validate the JSON type here. bool("false") is True, so a malformed-but-plausible reviewer response can incorrectly permit completion. Accept only an actual boolean, otherwise treat the verdict as incomplete or fall back safely.

@teknium1 teknium1 added sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 15, 2026

@GottZ GottZ left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was generated by AI during triage.

Summary

Two PRs address the same autonomous goal-chasing feature with effectively identical diffs: both add an engine-level continuation loop, independent completion review, clarify auto-answering, deception detection, an ADR decision log, CLI/TUI integration, and tests, but do so as a parallel orchestration engine rather than extending the existing persisted /goal path.

Related pull requests

  • #49917 [closed] duplicate — (+5425/-4) — superseded duplicate: This is the original version of the full autopilot implementation and remains relevant as the source of the initial design discussion, but it was recreated unchanged as #51565 solely to correct branch-based feature tagging.
  • #51565 related — (+5425/-4) — keep open with a salvage path: The diff contains salvageable independent-review, ADR logging, and notification-autodispatch work, but the contributor keep_open review identifies blocking architectural and safety issues: it duplicates the established /goal orchestration path, implicitly enables YOLO, continually extends budgets without a hard bound for tool-call-only loops, and persists Council-denied text into a profile-global substring matcher.

Duplicates

#49917 and #51565 are effectively the same code and diff; #49917 is the closed predecessor superseded by #51565.

Suggested consolidation

Keep #51565 open with a salvage path, consistent with the contributor keep_open review: reconcile the useful independent completion-review and ADR pieces with the existing /goal implementation, keep autonomous continuation separate from dangerous-command authorization, enforce a hard budget that also covers tool-call-only loops, and make learned deception signals run-scoped or explicitly curated rather than automatically profile-global. #49917 should remain closed as the superseded duplicate of #51565; no merge recommendation is supported for either PR in its current form.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    subgraph Dup49917 ["PRs duplicating each other"]
        P49917["PR #49917 (closed)"]
        P51565["PR #51565 (open)"]
    end
    class P49917 closed
    class P51565 open
    class P51565 target
    click P49917 "https://github.com/NousResearch/hermes-agent/pull/49917"
    click P51565 "https://github.com/NousResearch/hermes-agent/pull/51565"
Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label).

Cross-PR triage: Reviewed 2 pull requests and 0 issues in this complex. Each diff was read against this issue; Assessment working set: 506 kB of PR diffs, 27 kB of issue/PR text, 6 kB of discussion (5 comments), 0 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard comp/tui Terminal UI (ui-tui/ + tui_gateway/) P3 Low — cosmetic, nice to have sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants