feat(notifications): opt-in suppression of user-channel warning notifications (display.suppress_warning_notifications)

[victor-kyriazakos]

Summary

Opt-in suppression of automatic warning and diagnostic notifications that Hermes pushes into user channels (Slack, Telegram, Discord, Matrix, Signal, WeChat/Weixin, Google Chat, native Bot Chat, CLI/TUI), with a per-platform override:

display:
  suppress_warning_notifications: false   # default: legacy behavior, nothing suppressed
  platforms:
    slack:
      suppress_warning_notifications: true

Absent/false is byte-for-byte legacy. true suppresses automatic diagnostics only: compression/retry/fallback notices, credit-service and subagent failure notices, inactivity and watchdog stalls, media-delivery fallbacks, startup DB warnings, shutdown cron-interrupt notices, hygiene notices, cron failure_deliver alerts. It never touches requested results, approvals, clarifications, progress, cleanup, logs, telemetry, or failure bookkeeping — the producer runs unconditionally; only presentation is gated.

How it is wired

One resolver, a small set of typed boundaries, no rendered-text/prefix heuristics:

• warning_notifications_enabled(platform, user_config) — the single policy read (destination-owner/profile aware, per-turn snapshot).
• BasePlatformAdapter.emit_warning / emit_media_warning — async channel sends; None means suppressed (never a fabricated receipt); captions/requested parts always delivered; diagnostic sends never seal an unrelated active stream.
• warning_text(visible, hidden) — mixed adapter text where the requested part appears in both variants.
• render_notification / present_notification — sync (CLI/TUI) and async (lifecycle emitters) render boundaries.
• diagnostic_turn_muted — the one admission rule for diagnostic-category internal wakes across gateway/CLI/TUI/API.

Producers classify (notification_category); classification survives callbacks, threads, queues, retries, persistence/replay, ingress and streaming. Direct policy reads went from 56 scattered guards to the resolver plus the boundaries above.

Cron delivery ledger

Suppressed cron failure alerts are a durable disposition, not a send, so they exposed pre-existing ledger races. Fixed in this PR, each with its own regression cells:

• Incident occurrence generation + atomic binding inside the producer's upsert; native alerts fenced to the bound generation.
• Deferred (Bot Chat) delivery: pre-send manifest intent flag on the run row, cleared only by the manifest write; every exit after the mark settles a manifest; post-send ledger faults never surface as transport failure or trigger a resend.
• Reconciler: CAS on flag and exact manifest; jobs projection freshness (a slow reconciler can't resurrect queued after terminal settle); retention keeps rows until projection completes.
• Upgrade path: one-time atomic adoption of pre-flag journaled intent behind a schema_meta fence row (strict inventory, rolled back and retried on any fault); readers hold placeholder-only rows until adoption completes; per-row late-writer accounting for old processes that publish after the fence.
• finish_execution derives its terminal decision from the authoritative row and writes with a CAS on the exact manifest it derived from.

Disclosed, non-blocking follow-ups (not this PR): SQL instr('"bot"') vs parsed-key completeness differ for two synthetic manifest shapes no producer emits; rows whose manifest and journal persistence are both permanently lost stay pending unbounded by design (needs an operator dead-letter policy).

Verification

• Full tests/cron: 1,600 passed, 0 failed, 1 Windows-only skip. Refactor slice: 302 tests / 21 files; each shared boundary has a staging mutation that fails the expected cells when disabled.
• Area gate (gateway, cron, tui_gateway, hermes_cli, agent): 34,117 passed; the only failures were pre-existing and reproduced on baseline (_load_cfg_raw signature collision, dashboard port-in-use, updater state).
• Independent adversarial review (Salt), 10 rounds on frozen worktrees at exact heads; every reviewer probe is ported verbatim into tests/cron/test_r*_salt_*.py. Round 10 at 3224cb1c60: CLEAR for the refactor + cron remediation scope.
• Whole-stack 26-family acceptance refresh (Salt R11): one defect found and fixed (f5b807e7bd, sync delegate failure echo); remaining families verified at default-off byte parity.
• Code-flow grid sweep of every emission site not touched by this diff (3,796 candidate print/send/status/notice sites in the runtime tree, clustered into 31 code-flow segments, each classified by an independent lane and every reported gap re-verified by hand): 20 pre-existing automatic diagnostics were reaching users outside any boundary. All 20 now go through the shared boundary, each with tests that fail when the gate is removed — gateway shutdown/restart broadcast, unauthorized-sender owner hint, /bg crash notice, -Q linger admission, turn-1 compression-warning replay (lost its diagnostic class on the way to the gateway status sink), stream reconnect/stall markers, agent-init context-length and autoraise prints, tool-name auto-repair print, CLI session-store/browser-downgrade/provider-fallback/iteration-budget/vision-fallback notices, TUI model-adoption failure, /goal compression-recovery notice, TUI preview-restart status rows, Hindsight root-guard notice. Zero gaps in Slack, cron, relay/stream, API/base adapter, kanban, or any platform adapter. Area gate at the sweep head: 44,067 passed; every failure reproduced on baseline.

[victor-kyriazakos]

Reopening the scope review: this implementation gates selected gateway warning callbacks, not all automatic warning/failure notifications delivered to user channels. Existing cron, Kanban, background and adapter-owned paths need an explicit coverage/precedence review before this is ready.

A real-config, recording-transport probe against ef42c37 reproduces the session-stall watcher posting its warning with display.warning_notifications: false (gateway/run_watchers.py:_notify_session_stall). The separate per-worker inactivity warning is gated, so this is a missed sibling path. Adapter-owned upload notices also bypass the setting.

The lifecycle prefix tuple matches existing rendered messages; it does not add warning producers. However, it couples suppression to wording and is not a complete emission contract. A producer-to-destination audit is underway, including manual/scheduled cron execution, Kanban notify/wake delivery, background completions, replay, and native/relay paths.

The existing test and independent-review results remain valid for the tested subset. They do not establish global coverage or OTLP diagnostic parity. Converted back to draft pending that review; no merge/deployment claim.

[kshitijk4poor]

Thanks @victor-kyriazakos — this landed as a salvage in #114370 with your authorship preserved (your 54 commits are squashed into one commit authored by you, 28207a5277, so a rebase-merge keeps your name on it), plus follow-up commits from me.

What was kept: the whole suppression feature — display.suppress_warning_notifications (global + per-platform), the resolver, the adapter boundary (emit_warning / emit_media_warning / warning_text), the notification_category classification through wakes/queues/persistence, the CLI/TUI render boundaries, and the cron suppressed disposition.

What was split out (each is a separate PR if you want to pursue it; happy to review):

• The cron delivery-manifest ledger rework (manifests + journal, schema_meta fence, legacy-intent adoption, incident generations, jobs projection CAS, per-tick reconciler). origin/main has none of the state that layer migrates from — the "pre-flag old writer" the r5–r9 tests simulate is _r5_prior_executions.py, a snapshot of this PR's own earlier revision. The incident-generation fence and the jobs-projection CAS may fix real races; they need a main-reproducing test each.
• _interim_send on warn statuses + the Slack/relay seal guards: main's status sends carry no interim mark for any kind, so that's a class-wide streaming-contract fix (gateway/AGENTS.md rule 3) on its own.
• Running the whole post-handler delivery inside the routed profile scope: under multiplex it moved delivery-ledger writes to the routed state.db while boot-time recovery reads the launch home. Only the policy reads bind the routed scope in the salvage.
• The "delivery failed after multiple attempts" notice stays unconditional (the requested result was lost; that line is its only signal), and tests/cron/test_incident_binding_atomicity.py (git show <PR commit> would go red once a rebase-merge rewrote the SHA).

Closing this one in favour of #114370.