Skip to content

fix: suppress internal status banners on customer-facing platforms (refs #28208) - #30574

Open
ozzaii wants to merge 1 commit into
NousResearch:mainfrom
ozzaii:fix/emit-status-customer-facing-28208-partial
Open

fix: suppress internal status banners on customer-facing platforms (refs #28208)#30574
ozzaii wants to merge 1 commit into
NousResearch:mainfrom
ozzaii:fix/emit-status-customer-facing-28208-partial

Conversation

@ozzaii

@ozzaii ozzaii commented May 22, 2026

Copy link
Copy Markdown

Summary

Partial fix for #28208. The WhatsApp gateway (and any other customer-facing
platform) currently delivers Hermes-internal diagnostic banners straight
into end-user chat threads — for example:

🗜️ Context too large (~156,287 tokens) — compressing (1/3)...
🗜️ Compressed 120 → 7 messages, retrying...

Root cause: AIAgent._emit_status dispatches to the gateway adapter via
status_callback directly, which bypasses the transform_llm_output
plugin hook that sanitiser plugins typically attach to.

This PR adds a platform-aware filter to _emit_status:

  • New customer_facing: bool = True kwarg (backwards-compatible default).
  • New _CUSTOMER_FACING_PLATFORMS = {whatsapp, slack, signal, telegram, discord}.
  • New _INTERNAL_STATUS_PREFIXES defensive glyph fallback — kept narrow
    (compression-banner category only).
  • The four compression-recovery emit sites (1× 413 payload, 1× context-
    too-large, 2× compressed-retrying) now opt out explicitly with
    customer_facing=False.
  • CLI / admin / api surfaces still receive every status — operator
    visibility unchanged.

Why "partial"

#28208 also requests silent-success for empty responses, broader provider-
diagnostic suppression, and a whatsapp.allow_silent_response config flag.
Those are deferred to follow-ups so this PR stays small enough to review
and land quickly. The compression-banner scope was chosen because:

  • It is the one currently leaking into a real production customer chat
    (live incident 2026-05-22 in an AIRA-on-Hermes deployment), so the
    marginal value is highest.
  • It is strictly subtractive on customer-facing platforms — zero
    behavioural impact on CLI / API surfaces, no public-API breakage.

Test plan

  • tests/run_agent/test_emit_status_customer_facing_filter.py — 13
    new cases covering:
    • all five customer-facing platforms drop compression status when
      customer_facing=False
    • glyph fallback drops compression banners from un-migrated callers
    • real customer messages still reach the gateway on WhatsApp
    • explicit customer_facing=False drops even non-glyph messages
    • CLI / api_server / empty-platform surfaces always receive status
    • status_callback exceptions remain swallowed (fault-tolerance kept)
  • Existing test_empty_response_emits_status_for_gateway still passes.
  • Existing test_context_compression_triggered still passes.
  • No new dependencies; no public-API breakage (new kwarg has a default).

Out of scope (follow-up PRs)

Tracked under #28208 — happy to take any of these in this PR if the
maintainers prefer a wider scope:

  1. Silent-success path for empty model responses (config-gated).
  2. whatsapp.allow_silent_response: true / suppress_diagnostics: true
    config flag wiring.
  3. Broader internal-banner suppression beyond compression: empty-response
    retry (⚠️ Empty/malformed), rate-limit (⏱️/⚠️/❌), retrying (),
    max-retries (⚠️/❌), non-retryable (⚠️), provider-auth (⚠️),
    proxy (⚠️). My local deployment ships these in the glyph fallback
    for defensive coverage, but each touches an additional set of emit
    sites and felt big enough to deserve its own review.

…ousResearch#28208)

Compression-recovery emissions like "🗜️ Context too large (~156k tokens) —
compressing (1/3)..." were reaching end-user chat threads on WhatsApp,
Slack, Signal, Telegram and Discord because ``AIAgent._emit_status``
dispatches via ``status_callback`` directly, bypassing the
``transform_llm_output`` plugin hook that sanitiser plugins commonly
attach to.

Add a ``customer_facing: bool = True`` kwarg to ``_emit_status``. When the
runtime ``platform`` is in the customer-facing set and either:

* ``customer_facing=False`` is passed explicitly, or
* the message starts with an internal-status glyph in the (narrow,
  compression-only) defensive fallback list,

the gateway dispatch is skipped. CLI / admin / api surfaces still
receive every status through ``_vprint``, preserving full diagnostic
visibility for operators.

Annotates the four compression-recovery emit sites — 1× 413 payload,
1× context-too-large, 2× compressed-retrying — with the explicit
kwarg so future audits surface the policy in code instead of relying on
the glyph fallback.

Refs: NousResearch#28208 (partial fix — compression-banner category only; broader
"silent-success" / empty-response / retry-banner suppression and the
config-level ``whatsapp.allow_silent_response`` proposed in the issue
are deliberately out of scope here, to keep this PR small enough to
review and merge quickly).

Tests
-----
* ``tests/run_agent/test_emit_status_customer_facing_filter.py`` — 13
  cases covering all five customer-facing platforms, the glyph
  fallback, the explicit kwarg path, CLI / api admin visibility, and
  ``status_callback`` exception safety.
* Existing ``test_empty_response_emits_status_for_gateway`` and
  ``test_context_compression_triggered`` still pass — backward
  compatibility preserved for callers that omit the new kwarg.
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint platform/whatsapp WhatsApp Business adapter comp/gateway Gateway runner, session dispatch, delivery labels May 22, 2026
@jacobburrell

Copy link
Copy Markdown

This seems great, but it does seems to be separate from #18848 since AFAIK the gateway will nudge the model to spit out a response even if status banners aren't dumped into the platform.

Also I would like the option to see status banners elsewhere. Perhaps home channel or another interface so that admins or the owner of the hermes setup can be notified/see the logs or info lost from the status banner info.

Just not appropriate for non admins to see this.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for isolating a real customer-facing failure mode. The current main architecture has moved the relevant logic, so this needs a small re-scope rather than a direct cherry-pick.

Problems

  • Compression recovery now buffers status in agent/conversation_loop.py:3617 and agent/conversation_loop.py:3638; terminal failure flushes it at agent/conversation_loop.py:3646 through run_agent.py:1026-1029. The PR's old run_agent.py emit-site edits therefore no longer cover the live path.
  • Delivery policy is now centralized in gateway/run.py:440-457, with only programmatic surfaces retaining raw diagnostics (gateway/run.py:94-107). A five-platform allowlist in AIAgent would duplicate and narrow that policy.

Suggested changes

  • Extend the gateway noise classifier for the current Context too large … compressing and Compressed … retrying status forms, and add them to tests/gateway/test_telegram_noise_filter.py across chat and raw programmatic surfaces.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists platform/whatsapp WhatsApp Business adapter sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants