feat(autopilot): engine-enforced goal-chasing with a Council-or-fallback reviewer + decision log

[arminanton]

What /autopilot does

Autopilot is engine-enforced goal-chasing: tell it a goal and it keeps working until the goal is verifiably complete, instead of stopping at the first plausible "I think that's done." The mechanism that makes this trustworthy is an independent reviewer at every decision point. The agent never grades its own homework. A separate judge decides "is the goal genuinely complete, or must I keep going?", and auto-answers any clarify question with the most-defensible choice so an unattended run never dead-ends on a 3am "should I continue?" prompt.

This PR completes that design in two ways: it makes autopilot fully usable without the Hermes Council (so anyone can adopt it), and it gives every run a reviewable decision trail.

The reviewer: Hermes Council (preferred) with a graceful fallback

The preferred reviewer is the Hermes Council, and it is worth explaining why, because the Council is the part that makes autonomous loops actually reliable rather than fragile.

A single model reviewing its own work tends to agree with itself. That self-agreement is exactly what lets a lazy "done" slip through and what makes naive agent loops either stop too early or loop forever. The Council replaces that with structured disagreement: multiple independent critic personas that are explicitly told not to balance or be agreeable, and a separate Arbiter that reads their raw outputs as data, separates "what the user wants" from "what the evidence supports" from "the safest reversible path," and applies an accuracy ceiling that caps confidence when evidence is thin or the critics disagree. For a completion gate this is the right axis to optimize: the Skeptic's whole job is to find why the work is not done, so a sycophantic "complete" has to survive an adversarial pass before autopilot is allowed to stop.

The genuinely novel part, and the reason this is worth adopting even on a single subscription: the Council runs all of this from one model. You don't need multiple providers or API keys. It convenes 24 anti-sycophancy personas as separated roles/profiles over whatever model Hermes already has configured (Copilot, Codex, OpenRouter, direct keys) via its Hermes-native in-process lane. Multi-provider diversity is an optional enhancement, not a requirement. There's a durable daemon that owns each run (re-attachable, single-flight dedup, self-recovery reaper with per-call wall-clock bounds), anonymous peer review (Borda ranking + a disagreement map), and a sycophancy audit that can force a Devil's-Advocate re-deliberation. If you want the strongest version of autopilot, run the Council:

https://github.com/arminanton/hermes-council

A note for anyone who tried the earlier hermes-council and moved on: this is not that project. It's a ground-up rebuild that keeps the original MCP contract (same tools, fast/standard/deep modes, structured verdicts) but is a different engine underneath. The original shipped 5 personas as a pip package; this one is a self-contained project with 24 personas across 9 bounded panels and 6 presets, an engine-enforced accuracy ceiling, anonymous peer review, a sycophancy audit, a durable re-attachable daemon with per-call wall-clock bounds, multi-lane providers (hermes-native in-process, codex-cli, copilot-cli, gemini-cli, offline), and a benchmark harness (Council-vs-single-model on a planted-flaw dataset). If your mental model of "hermes-council" is the old 5-persona version, the behavior here is materially different. Worth a fresh look.

Point Hermes at it with COUNCIL_PROVIDER=hermes and it reuses your existing model and auth; no extra subscription.

When the Council is not installed, autopilot degrades cleanly: first to a single independent auxiliary-model reviewer pass (still not the main model grading itself), and finally fails open (stop) rather than looping blindly. This PR adds the missing piece of that fallback so it matches the documented design: when there's no Council, the clarify auto-answer now surfaces the full set of options it weighed and the single recommended pick (via choose_answer_detailed), rather than choosing silently, and labels which reviewer produced the decision.

The memory that makes long runs survivable: hermes-cmx

A reviewer keeps autopilot honest about whether it is done. But there's a second failure mode that quietly kills long autonomous runs, and it is about memory, not judgment.

An unattended goal-chase doesn't run for ten turns. It runs for hundreds, often well past a thousand, and every turn adds tool output, file reads, and reasoning to the history. Long before the goal is met, the conversation outgrows any context window. The usual response is to summarize the old turns into a paraphrase and move on, and that is exactly where a long run starts to rot: the agent reads its own lossy summary, loses the actual goal text and the decisions it already made, and confidently continues from a plausible-but-wrong reconstruction. It re-does finished work, contradicts an earlier decision, or drifts off the goal entirely, and because no human is watching, nobody catches it until the run is hours of wasted effort deep. A completion reviewer can't save you here, because the thing being verified is already built on a hallucinated memory.

hermes-cmx removes that failure mode at the root. It is a context engine for Hermes that keeps every message verbatim, forever, in an append-only store, and on every turn retrieves the exact slices the model needs (hybrid FTS5 + trigram + embeddings, fused and packed to the model's real window) instead of trusting a summary. The goal you set on turn 1 is still retrievable word-for-word on turn 1,300. The decision you made 400 turns ago comes back verbatim, with a citation, the moment it's relevant again. And it doesn't stop at retrieval: cmx verifies the model's answer against the store and refuses ungrounded claims, so an autonomous run can't quietly invent a fact about its own history and then act on it.

This is what turns "keep working until the goal is verifiably complete" from an aspiration into something that actually holds over a long horizon. The proof numbers are concrete and reproducible (see the cmx repo's benchmarks/results/):

• 1,050,928 tokens ingested across 588 turns in ~3 seconds, planted sentinel facts retrieved verbatim after the full conversation;
• an 8,000-token window answering correctly over 663-turn conversations (6×+ larger than the window) at 76.5% accuracy, because the memory lives in the database, not the window;
• 0.0% shipped hallucination vs 10.0% for the summarize-and-hope baseline on identical questions.

In short: the Council keeps an autonomous run from stopping on a lazy "done"; cmx keeps it from forgetting what "done" even means. Autopilot is most reliable with both, the reviewer guarding the exit and cmx guarding the memory. cmx is optional (autopilot runs without it), but for genuinely long unattended runs it is the difference between an agent that remembers its whole mission and one that paraphrases it away around turn 200.

Both pillars share the same design philosophy as autopilot itself: don't trust the model to behave, build the guarantee into the engine. The reviewer enforces honesty about completion; cmx enforces honesty about memory.

The decision log (ADR)

The moments autopilot makes a call (complete vs keep-going, and how it answers a clarify) are exactly the moments a human would normally be in the loop. So when autopilot.adr is enabled, every one of those decisions is appended to a human-readable markdown file you can review after an unattended run:

• what was sent for verification (the goal, the candidate result, the work context),
• what the reviewer returned: verdict, confidence, the specific gap it found, and the exact checks it said were required to reach a passing state,
• the options on the table and which path autopilot took, with a one-line rationale.

With the Council the record is rich, because the Council returns why a completion claim fails and what to verify to fix it, so the ADR captures both the submission and the structured verdict. In the fallback lane it records the options plus the recommended choice. Either way you get a lossless, append-only audit trail of every autonomous decision. It's off by default, writes only a local file under .hermes/, and fails soft (an ADR error can never break a run).

Changes

• agent/autopilot/adr.py: new append-only decision-log module (enabled via autopilot.adr / HERMES_AUTOPILOT_ADR, path overridable via autopilot.adr_path / AUTOPILOT_ADR_PATH).
• agent/autopilot/council_gate.py: choose_answer_detailed() returns the answer plus the options weighed, a rationale, and the reviewer source; choose_answer() stays a thin string wrapper so existing callers are unchanged.
• agent/autopilot/driver.py: ADR records written at the two decision points (maybe_continue completion + continue branches, and the clarify callback).
• cli.py / hermes_cli/config.py: autopilot.adr + autopilot.adr_path config keys bridged to the env vars the driver reads.
• hermes_cli/cli_agent_setup_mixin.py: re-apply the session's autopilot state to the agent when it is rebuilt on a model/route switch, so a /autopilot toggle survives an in-session model change.
• TUI: a 🤖 AUTO status-bar badge for the autopilot session field (the companion YOLO badge ships separately).

Tests

tests/agent/test_autopilot_adr.py (11): default-off, enable via attr/env, append-not-overwrite, options/gap/required-checks captured, fail-soft on a bad path, path override, default path shape. Plus options-surfacing tests in the council-gate suite (3), ADR-wiring tests in the driver suite (4), and the AUTO-badge TUI tests (2). The full autopilot suite is green (81 passed, 1 skipped where the offline Council package isn't present).

Note on scope

This PR also folds in the autopilot 🤖 AUTO status badge and keeps the notify_autodispatch gate that belongs to the autopilot subsystem. It's one self-contained feature: engine-enforced autopilot with a Council-or-fallback reviewer and a reviewable decision log. The Council (reviewer) and hermes-cmx (memory) are both optional companions, linked above; autopilot runs without either, but they are what make long unattended runs trustworthy.

[maxonliu]

This is exactly the feature I was going to open an issue for — the Claude /loop equivalent for Hermes. Glad to see it's already in the works.

Design looks solid. The cooperating + enforced split is the right call. Telling the model "you're in autopilot" gets you 80% of the way through system-prompt nudging, but having an independent reviewer to catch fake completions closes the gap that makes autonomous loops fragile in practice. The Council reviewer (v2) is a nice touch — anti-sycophancy is exactly the right axis to optimize for here.

Real use cases this unlocks:

• /autopilot goal "fix all lint errors in this project" — scan → fix → verify, multi-pass until zero warnings
• /autopilot goal "refactor this module and make tests pass" — iterative TDD without babysitting
• Overnight cron jobs that don't dead-end on a "should I continue?" prompt at 3am

Complementary to #21172 (loop contracts). This PR handles the intra-session "don't stop" behavior; #21172 handles the inter-session governance layer — budget, scope, stop conditions for cron-backed loops. They compose cleanly: autopilot runs the loop, the contract keeps it from burning money.

One question on the design: does /autopilot goal <text> also set the verification target for the reviewer, or is the goal inferred from the conversation? Explicit seems safer for unattended runs, but I may be missing how goal flows into the review gate.

Looking forward to seeing this land. Happy to help test when ready.

[arminanton]

@maxonliu thanks, and good question on the goal flow. It's explicit, by design, for exactly the unattended-safety reason you flagged.

/autopilot goal "<text>" sets agent._autopilot_goal, and that exact string is what resolve_goal() hands to the reviewer as the verification target (judge_completion(goal, work_summary, candidate_result)). It is not re-derived from the conversation. If no explicit goal is set it falls back to the active standing goal, and only then to inferring from the session, but for an unattended run you'd always set it explicitly, so the gate judges against the thing you actually asked for, not a paraphrase of it.

On the reviewer itself: you're right that the anti-sycophancy is the crux, and it's the Hermes Council that provides it (it isn't native to autopilot, autopilot just drives it). The Council is the independent panel that has to fail to refute a completion claim before autopilot is allowed to stop, so a lazy "done" doesn't pass. I just pushed a follow-up that makes this usable for people who don't run the Council yet: when it's absent, autopilot falls back to a single independent reviewer pass, and every decision (what was sent for verification, the verdict, the gap found, the required checks) is written to a markdown ADR log you can review after the run. So the goal to gate flow is the same whether or not you have the Council; the Council just makes the verdict much harder to fool. Happy to have you test it when you're ready.

[arminanton]

Superseded by #51565, recreated with a correct feat/ branch name so it's tagged as a feature rather than mis-labeled type/bug (the old branch fix/tui-notify-autodispatch-gate triggered the bug auto-tag). Same code, same diff, same author. The goal-flow design discussion above with @maxonliu still applies to the new PR.