Skip to content

feat(context-engine): post-response grounding-enforcement hook - #50155

Draft
arminanton wants to merge 1 commit into
NousResearch:mainfrom
arminanton:feat/cmx-grounding-enforcement-hook
Draft

feat(context-engine): post-response grounding-enforcement hook#50155
arminanton wants to merge 1 commit into
NousResearch:mainfrom
arminanton:feat/cmx-grounding-enforcement-hook

Conversation

@arminanton

Copy link
Copy Markdown
Contributor

Summary

Adds the host-side grounding-enforcement seam that a grounding-aware context
engine plugs into. After the assistant message is normalized in the conversation
loop, if the active context engine advertises an enforce_response capability
(via a capabilities() method returning {"enforce_response": True, ...}), the
engine may audit the final answer against its record and return a replacement
when the answer is ungrounded.

Design

  • No-op by default. The built-in ContextCompressor and LCM have no
    capabilities() method, so they are entirely unaffected.
  • Duck-typed. Uses getattr(_eng, "capabilities", None) + a callable check,
    so any engine lacking the API is silently skipped — no hard dependency on a
    particular engine being installed.
  • Crash-safe. The whole block is wrapped; enforcement can never break a turn.
  • Final text only. Skips tool-call turns and empty content.

This is the single host-side seam for a verbatim-memory / grounding context engine.
The engine implementation itself lives in its own component; this PR is just the
provider-agnostic hook in the conversation loop.

Tests

tests/agent/test_grounding_enforcement_hook.py (6 cases): replace verdict,
keep verdict, engine without capabilities() (no-op), no engine (no-op),
tool-call turn (skipped), empty content (skipped). All pass; ruff clean.

Built on origin/main; applies onto v0.17.0.

Adds an optional, duck-typed grounding-enforcement seam to the conversation
loop: after the assistant message is normalized, if the active context engine
advertises an "enforce_response" capability (via a capabilities() method), the
engine may audit the final answer against its record and return a replacement
when the answer is ungrounded.

No-op for the built-in ContextCompressor and LCM (they have no capabilities()),
duck-typed via getattr so any engine lacking the API is skipped, and fully
wrapped so enforcement can never break the turn. This is the host-side seam a
grounding-aware context engine (e.g. a verbatim-memory engine) plugs into.

6 tests cover replace/keep verdicts, no-capabilities engine, no engine,
tool-call turns (skipped), and empty content (skipped).
arminanton added a commit to arminanton/hermes-agent that referenced this pull request Jun 21, 2026
…v0.17 evidence

- CMX grounding-enforcement hook graduated from deferred to its own draft PR
  NousResearch#50155 (the single CMX-implementation PR per user rule). Only the private-path
  schema test remains deferred (cannot be public).
- v017_per_pr.txt: per-PR v0.17.0 re-applicability for all 39 feature PRs
  (37 clean + 2 --3way with forward-compat branches + 0 conflict).
- symdiff_cleanclone_output.txt refreshed: PASS with 39 PR heads.
@alt-glitch alt-glitch added type/feature New feature or request comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have labels Jun 21, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Related: #50053 (the additive grounding hook points — capabilities()/pre_send/enforce_response — on the base ContextEngine ABC in agent/context_engine.py). This PR adds the complementary host-side call site in agent/conversation_loop.py that invokes enforce_response after the assistant message is normalized. Same author, different layer (ABC definition vs conversation-loop invocation) — companion, not a duplicate.

@arminanton

Copy link
Copy Markdown
Contributor Author

Thanks @alt-glitch — correct, this is the conversation-loop call site invoking enforce_response after the assistant message is normalized, the companion to #50053's ABC hook-point definitions. Same author, different layer; complementary not duplicate.

arminanton added a commit to arminanton/hermes-agent that referenced this pull request Jun 22, 2026
…open questions

Addresses the Council demand for platform/review state, not just local repro.

Pulled directly from GitHub for all 42 PRs (PER-PR-PLATFORM-STATUS.txt):
- 42/42 OPEN (8 ready-for-review, 34 draft).
- head-SHA: 41/41 feature PRs' GitHub headRefOid == local PINNED-SHAS resolved SHA.
  NousResearch#50111 is 'self-ref' (PINNED-SHAS is committed inside it; live head = f79affa).
- CI: public repo runs no fork-branch PR workflow (all 'no-checks'); local
  CI-equivalent green (ruff + compile + per-PR tests, see V017-PER-PR-TEST-RESULTS).
- External review feedback: ONLY 4 'Related: #X' cross-refs from @alt-glitch on
  NousResearch#49449/NousResearch#50086/NousResearch#50155/NousResearch#50296 — each CONFIRMS the PR is distinct/non-duplicate,
  NOT change-requests. 0 unaddressed review threads, 0 submitted reviews.
- The 6 conflict PRs are MERGEABLE on their own base (origin/main); the v0.17.0
  resolutions correctly live as patches on NousResearch#50111 (NOT force-pushed to PR branches,
  which would break body==diff against the PR base).

Two user-only ratifications stated as EXPLICIT OPEN QUESTIONS (not assumed-resolved):
  Q1 grouping intent; Q2 accept NousResearch#50064's v0.17.0 forward-compat test drop (NousResearch#2647).
arminanton added a commit to arminanton/hermes-agent that referenced this pull request Jun 22, 2026
…rdict

Each open PR replayed onto v0.17.0 + tests run. Verdict per PR (not just coverage):
34 clean apply + tests/code-only; 6 conflict PRs with verified v0.17.0 patches; the
5 with failures root-caused (NousResearch#50066/NousResearch#50086 pre-existing v0.17.0 flake reproduced on
clean v0.17.0; NousResearch#50031/NousResearch#50032/NousResearch#50078 declared stack-deps that pass on full overlay).
NousResearch#50146 + NousResearch#50155 (secondary co-owners, previously only co-listed) now individually
verified: CLEAN apply + 6 passed each. 0 PRs left needing review.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the focused companion host hook. The underlying extension point is not on current main, so this remains useful work, but the draft needs relocation to match the current conversation-loop finalization path.

Problems

  • The proposed post-normalization call is not final-only on current main: agent/conversation_loop.py:4387-4431 handles Codex finish_reason == "incomplete" after normalization and continues the turn. Calling the engine there with final=True can audit an intermediate response.
  • Final text is assembled later: agent/conversation_loop.py:5137-5142 joins truncated parts and strips think blocks, and :5268 appends the final message. The proposed call would audit a fragment rather than the delivered result.
  • tests/agent/test_grounding_enforcement_hook.py:14-30 mirrors the production block instead of invoking the conversation loop, so it cannot verify integration placement or persistence.

Suggested changes

  • Rebase the implementation concept onto the current final-response path after final_response is composed at agent/conversation_loop.py:5137-5142 and before final-message construction/persistence.
  • Add real run_conversation() coverage for replacement, incomplete-response skip, and length-continuation composition.
  • Coordinate the capability/verdict API with companion PR #50053, which is still open.

Automated hermes-sweeper review.

_has_tools = bool(getattr(assistant_message, "tool_calls", None))
_caps = getattr(_eng, "capabilities", None)
if (_eng is not None and not _has_tools
and isinstance(_content, str) and _content.strip()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This runs immediately after normalization, but a normalized response is not necessarily final: current main continues Codex finish_reason == "incomplete" responses later in the loop. Move enforcement to the final-response path after continuation/recovery and response-part assembly, otherwise final=True can replace an intermediate answer.

import types


def _apply_enforcement(agent, assistant_message, messages):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This helper mirrors the implementation, so these tests do not exercise the conversation-loop call site. Please drive run_conversation() with an opt-in fake engine to verify replacement is returned and persisted only for a completed final response.

@teknium1 teknium1 added sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users labels Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants