Repository navigation
feat(agent): consume the out-of-band hermes_confab_notice response extension - #764
Merged
Merged
Conversation
…tension A bridge that catches and strips self-fabricated scaffold text from a model reply must still tell the user — that signal is load-bearing for triage. It is currently delivered IN BAND, appended to assistant content, which mutates conversation history and is replayed upstream on any full-history recovery. Move the consumer side to a versioned response-envelope extension, per claude-bpx docs/SPEC-confab-marker-out-of-band.md v1. Consumer-first is mandatory: this lands before any producer emits the field. With today's bridge (no extension field) behavior is unchanged. - agent/confab_notice.py: the single validation gate. Fails closed on unknown version, wrong kind, bad scope, non-string request_id, oversized labels. Returns a fresh dict of only the five contract keys, so a provider cannot smuggle extra fields into display_metadata. - chat_completion_helpers._call_chat_completions: scan chunks for the extension (contract puts it on the final usage chunk), accept at most one per response, forward it on the synthetic completion. - ChatCompletionsTransport.normalize_response: same extraction for non-stream completions, preserved in NormalizedResponse.provider_data. - conversation_loop: emit CONFAB_NOTICE_TEXT once per notice (request_id ledger stops a retry/fallback double-emit) via _emit_status, which reaches CLI, TUI and gateway. - build_assistant_message: stamp display_kind='confab_notice' plus versioned display_metadata on the assistant row. Content is never touched. - CLI resume recap + desktop display_kind union render the new tag. Replay exclusion is an existing invariant, not a promise. Pinned by symbol (AST walk of api_msg.pop) rather than line number, since the spec's own line citation moved twice. Verified: - tests/agent/test_confab_notice.py 42 passed - tests/agent/test_confab_notice_e2e.py 8 passed (stream + non-stream, real agent loop against an in-process mock provider) - tests/agent/test_confab_notice_replay_strip.py 5 passed - tests/tui_gateway/test_confab_notice_render.py 5 passed - mutation proof 1: delete the two api_msg.pop lines -> 2 AST pins + both wire e2e tests go red (5 failed, 8 passed); restored. - mutation proof 2: delete "display_metadata" from the flush row dict -> persistence pin + both persistence e2e tests go red (3 failed, 6 passed); restored. - neighbor suites green: tests/agent/transports (18 files, 420 passed), streaming + display (9 files, 127 passed), CLI resume/gateway history (6 files, 244 passed).
Collaborator
Author
FleetReviewConfidence: 3/5 Findings
FleetReview provenance · models: B=gpt-5.6-sol, C=claude-code-opus-5, F=gpt-5.6-sol, G=grok-4.6 · cost: $35.76 · duration: 1h 12m 29s · rounds: 2 · files examined: 11 |
Kyzcreig
added a commit
that referenced
this pull request
Sep 24, 2026
Re-port #779 onto current main without reverting #764 validation or #787 usage accounting. Carry fixed kind-specific labels and metadata-only history events; share the dropped-call retry budget and strip ephemeral correction pairs at finalization. Verified: 160 confab/CLI/gateway tests, 42 dropped-call and usage tests, 33 finalizer/status tests; git diff --check.
Kyzcreig
added a commit
that referenced
this pull request
Sep 24, 2026
Re-port #779 onto current main without reverting #764 validation or #787 usage accounting. Carry fixed kind-specific labels and metadata-only history events; share the dropped-call retry budget and strip ephemeral correction pairs at finalization. Verified: 160 confab/CLI/gateway tests, 42 dropped-call and usage tests, 33 finalizer/status tests; git diff --check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Hermes-side consumer for the out-of-band confab-catch notice defined in
claude-bpx
docs/SPEC-confab-marker-out-of-band.mdv1.A bridge that detects and strips self-fabricated scaffold text from a model reply
must still tell the user — that signal is load-bearing for triage. Today it is
delivered in band, appended to assistant
content, which mutates conversationhistory and gets replayed upstream on any full-history recovery. This PR moves the
notice to a versioned response-envelope extension: a status event for the current
turn, and presentation-only metadata on the assistant row for historical triage.
Consumer-first is mandatory per the spec — this lands before any producer emits
the field. With today's bridge (no extension field) behavior is unchanged, and
that is pinned by a test.
Transport contract
{"hermes_confab_notice": {"version": 1, "kind": "scaffold_confab_removed", "request_id": "3b264082", "scope": "visible", "grammar": "inbound"}}Non-stream: top-level on the completion. Stream: on the final usage chunk
(
choices: []).Changes
agent/confab_notice.pykind, badscope, non-string/empty/oversizedrequest_id, badgrammar. Returns a fresh dict of only the five contract keys, so a provider cannot smuggle extra fields intodisplay_metadata.agent/chat_completion_helpers.pybuild_assistant_messagestampsdisplay_kind='confab_notice'+ versioneddisplay_metadata.agent/transports/chat_completions.pyNormalizedResponse.provider_data.agent/transports/types.pyNormalizedResponse.confab_noticeaccessor, alongside the existingcodex_*/reasoning_*ones.agent/conversation_loop.py_emit_status(CLI + TUI + gateway). Arequest_idledger stops a retry/fallback re-normalizing the same response from double-emitting.hermes_cli/cli_agent_setup_mixin.py,apps/desktop/src/types/hermes.tsdisplay_kindin the CLI resume recap and the desktop type union.Assistant
contentis never touched — no suffix, no prefix.Replay exclusion
Both presentation fields are already stripped from every outgoing copy in
conversation_loop'sapi_msgbuilder. The spec's own line citation for thatstrip moved twice, so the new contract test pins it by symbol — an AST walk
for
api_msg.pop("display_kind"/"display_metadata")inside the function that alsocalls
_clone_message_for_send— plus a wire-level e2e asserting the notice neverappears in outgoing
messages[].Per spec §consumer step 4 the flush is not modified; the existing
display_metadatapass-through is asserted instead, so a future refactor thatdrops it fails loudly.
Verification
The e2e suite drives the real agent loop against an in-process mock provider
returning the v1 object in both wire shapes, and asserts on the captured
outgoing request bytes, not on internal state.
Mutation proofs (the tests are not vacuous):
api_msg.poplines → both AST pins and both wire e2e tests gored (
5 failed, 8 passed). Restored."display_metadata"from the flush row dict → the persistence pin andboth persistence e2e tests go red (
3 failed, 6 passed). Restored.Neighbor suites, all green:
Not in this PR
The bpx producer. It is a separate card, blocked on this one — the bridge must not
emit the field until this consumer is deployed.