Skip to content

fix(agent): reword help guidance to avoid Z.AI WAF prompt-block false 429 - #59958

Draft
fadhlillah2 wants to merge 1 commit into
NousResearch:mainfrom
fadhlillah2:fix/zai-waf-help-guidance-reword
Draft

fix(agent): reword help guidance to avoid Z.AI WAF prompt-block false 429#59958
fadhlillah2 wants to merge 1 commit into
NousResearch:mainfrom
fadhlillah2:fix/zai-waf-help-guidance-reword

Conversation

@fadhlillah2

Copy link
Copy Markdown

What does this PR do?

Z.AI's coding-plan endpoint (https://api.z.ai/api/coding/paas/v4, provider zai, model glm-5.2) deterministically rejects any request whose system prompt contains the exact literal You run on Hermes Agent, returning a misleading HTTP 429 / code 1305 "The service may be temporarily overloaded". Since HERMES_AGENT_HELP_GUIDANCE ships that literal in every assembled system prompt, every Hermes turn against Z.AI fails 3/3 retries and surfaces to users as a bogus rate-limit.

This PR rewords the guidance to You are running on Hermes Agent (by Nous Research), which passes the filter while keeping the identity statement (and the rest of the constant) byte-identical.

Controlled evidence (2026-07-07, same key / endpoint / model / payload shape, non-streaming chat/completions; each probe run twice):

System prompt contains Result
You run on Hermes Agent (by Nous Research) 429 / code 1305, both runs
You are running on Hermes Agent (by Nous Research) 200, both runs
Blocked literal placed in an assistant history message instead of system 200 (filter matches system prompt only)
Full default-assembled 24 KB system prompt (reworded) + 109 k-token real conversation 200
Synthetic 72 k-token payload without the literal 200 (payload size is not a factor)

Two additional data points worth flagging for maintainers: in the same probe session, bare Hermes Agent and the literal skill_view(name='hermes-agent') both passed — i.e. the blocked-literal set has drifted since #47685 and #56816 were bisected. The filter is opaque and changes over time, which is why this PR deliberately stays minimal (remove today's active trigger at the source, zero conditional logic, benefits every provider) and is complementary to the provider-gated chokepoint sanitizer proposed in #53006, which remains the right long-term defense.

Note for operators: Hermes persists the assembled prompt per session (sessions.system_prompt in state.db) and replays it verbatim on resume, so sessions created before this fix keep the blocked literal until they are recreated — or migrated in place:
UPDATE sessions SET system_prompt = REPLACE(system_prompt, 'You run on Hermes Agent', 'You are running on Hermes Agent');
(We hit exactly this in production: 462/483 persisted sessions still failed after patching the source.)

Related Issue

Related: #47685, #53002, #56816. Complements (does not replace) #53006.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • agent/prompt_builder.py: reword HERMES_AGENT_HELP_GUIDANCE opening from You run on Hermes Agent (by Nous Research). to You are running on Hermes Agent (by Nous Research). — one line; the rest of the constant is unchanged.
  • tests/agent/test_prompt_builder.py: add TestGuidanceConstants::test_help_guidance_avoids_zai_waf_blocked_literal — asserts the blocked literal is absent while the identity statement is preserved, so the trigger can't silently return in a future edit.

How to Test

  1. Unit tests:
    pytest tests/agent/test_prompt_builder.py tests/agent/test_system_prompt.py tests/agent/test_system_prompt_restore.py -q
    (178 passed locally.)
  2. Live reproduction against Z.AI Coding Plan (fails on main, passes with this PR; needs a GLM Coding Plan key):
    from openai import OpenAI
    client = OpenAI(api_key="REDACTED", base_url="https://api.z.ai/api/coding/paas/v4")
    for phrase in ("You run on Hermes Agent (by Nous Research).",
                   "You are running on Hermes Agent (by Nous Research)."):
        try:
            r = client.chat.completions.create(
                model="glm-5.2", max_tokens=8,
                messages=[{"role": "system", "content": f"You are a helpful assistant. {phrase}"},
                          {"role": "user", "content": "Reply with exactly: pong"}])
            print(repr(phrase[:20]), "->", r.choices[0].message.content)
        except Exception as e:
            print(repr(phrase[:20]), "->", e)
    # main's phrasing -> Error code: 429 {'error': {'code': '1305', ...}}
    # this PR's phrasing -> pong
  3. End-to-end: point a zai/glm-5.2 profile at the coding-plan endpoint and send any message; on main every turn fails after 3 retries with the misleading 429.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate (closest: fix(agent): sanitize "Hermes Agent" prompt block on Z.ai endpoints (#47685 follow-up) #53006 — chokepoint sanitizer, provider-gated; this PR removes the currently-active literal at the source and is intentionally complementary)
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass — ran the three prompt-related test files (178 passed); full suite not run locally
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: Ubuntu 24.04 (WSL2) + production Ubuntu VPS running the reworded prompt against Z.AI since 2026-07-06

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — N/A (no docs reference the literal; verified via git grep)
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — N/A (string constant only)
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A

Screenshots / Logs

Production failure signature on main (gateway log, before the reword):

WARNING agent.conversation_loop: API call failed (attempt 1/3) error_type=RateLimitError
  provider=zai base_url=https://api.z.ai/api/coding/paas/v4 model=glm-5.2
  summary=HTTP 429: The service may be temporarily overloaded, please try again later
  error=Error code: 429 - {'error': {'code': '1305', 'message': 'The service may be temporarily overloaded, please try again later'}}

Same session, after the reword: API call #1: model=glm-5.2 provider=zai in=107716 out=33 latency=11.6s → 200.

… 429

Z.AI's coding-plan endpoint (api.z.ai) deterministically rejects any
request whose system prompt contains the exact literal
"You run on Hermes Agent" with a misleading HTTP 429 / code 1305
"temporarily overloaded" body, making glm models unusable with the
default prompt. Rewording to "You are running on Hermes Agent" passes
the filter while keeping the identity statement intact.

Complements the provider-gated sanitizer proposed in NousResearch#53006: this
removes the currently-active blocked literal at the source for all
providers, with no conditional logic.

Related: NousResearch#47685 NousResearch#53002 NousResearch#56816
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/zai ZAI provider P2 Medium — degraded but workaround exists labels Jul 7, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for isolating a narrow source-level trigger. Current main still contains the targeted literal at agent/prompt_builder.py:141, and agent/system_prompt.py:198 appends this guidance to every newly assembled system prompt.

Problems

  • This does not remediate existing sessions: agent/conversation_loop.py:330-334 restores a stored prompt verbatim, and agent/conversation_loop.py:861-870 sends that cached prompt. Sessions created before the change can therefore retain the blocked text after upgrade.

Suggested changes

  • Either state that this is a fresh/rebuilt-session mitigation, or cover resumed sessions with a Z.AI-only outbound request-copy transformation that leaves the cached and persisted prompt byte-stable.
  • If expanding scope, add a resumed-session regression test for that boundary.

Automated hermes-sweeper review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists provider/zai ZAI provider sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants