Skip to content

fix(agent): inject tool-use-enforcement guidance model-driven, not to… - #53857

Open
tauros1983 wants to merge 1 commit into
NousResearch:mainfrom
tauros1983:fix/telegram-no-tools-stall
Open

tauros1983 wants to merge 1 commit into
NousResearch:mainfrom
tauros1983:fix/telegram-no-tools-stall

Conversation

@tauros1983

Copy link
Copy Markdown
Contributor

…ol-driven

Telegram session 20260628_003118_249c66d5 stalled after one assistant turn on z-ai/glm-5.2: the system prompt was 20,401 chars and contained NONE of 'Tool-use enforcement' / 'Execution discipline' / 'Parallel tool calls' markers. The model emitted its tool intent as a literal bash markdown code block (different syntax than the earlier [TOOL_CALL] markers but identical underlying failure mode), the runtime saw tool_calls=None, and the conversation loop ended with finish_reason=stop after one turn.

Root cause: the tool-use-enforcement block in
agent/system_prompt.py was wrapped in 'if agent.valid_tool_names:' so the entire guidance was skipped when the session happened to have no tools loaded. Telegram/Discord gateway entry points commonly start without tools, so every GLM/Qwen/DeepSeek session arriving through those channels got zero guidance.

The fix decouples the guidance injection from valid_tool_names: the model-name gate (TOOL_USE_ENFORCEMENT_MODELS substring check) now drives whether the block lands, independent of whether the current session has tools loaded. The valid_tool_names check is preserved only for:

  • Google-operational sub-block (content is about file paths and edit commands — irrelevant without tools)
  • parallel_tool_call_guidance (no point steering a no-tool session to batch calls)
  • task_completion_guidance (no point steering a session with no work to 'finish')

The model needs the instruction regardless of whether the current session happens to have tools loaded — a no-tools GLM session today may turn into a tools session tomorrow, and even within a single session the model needs to know it should emit structured tool_calls when those become available rather than falling back to markdown.

This is independent of PR #35087 (which widens the OPENAI_ gate to cover glm/qwen/deepseek via a config-driven tuple) — that PR was open for 28 days before this issue surfaced. PR #35087 addresses the wrong-family gate; this PR addresses the wrong-gate wrapper. Both are needed; both can land in any order.

Files:
agent/system_prompt.py +49/-22
tests/agent/test_system_prompt.py +103/-0

Tests: 14/14 pass in test_system_prompt.py, 170/170 pass in test_system_prompt.py + test_prompt_builder.py. ruff + ty clean. 9 new regression tests in TestToolUseEnforcementInjectionWithoutTools cover glm/qwen/deepseek positive, opus negative, parallel-call still tools-gated, enforce-off respected, explicit-on escape hatch, and google-operational block still tools-gated.

E2E confirmation: after merge, send a fresh Telegram prompt with z-ai/glm-5.2 and verify the assistant turn is followed by tool execution instead of a 1-turn stall.

What does this PR do?

Related Issue

Fixes #

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

How to Test

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform:

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

For New Skills

  • This skill is broadly useful to most users (if bundled) — see Contributing Guide
  • SKILL.md follows the standard format (frontmatter, trigger conditions, steps, pitfalls)
  • No external dependencies that aren't already available (prefer stdlib, curl, existing Hermes tools)
  • I've tested the skill end-to-end: hermes --toolsets skills -q "Use the X skill to do Y"

Screenshots / Logs

…ol-driven

Telegram session 20260628_003118_249c66d5 stalled after one assistant
turn on z-ai/glm-5.2: the system prompt was 20,401 chars and contained
NONE of 'Tool-use enforcement' / 'Execution discipline' / 'Parallel
tool calls' markers. The model emitted its tool intent as a literal
bash markdown code block (different syntax than the earlier
[TOOL_CALL] markers but identical underlying failure mode), the
runtime saw tool_calls=None, and the conversation loop ended with
finish_reason=stop after one turn.

Root cause: the tool-use-enforcement block in
agent/system_prompt.py was wrapped in 'if agent.valid_tool_names:'
so the entire guidance was skipped when the session happened to have
no tools loaded. Telegram/Discord gateway entry points commonly
start without tools, so every GLM/Qwen/DeepSeek session arriving
through those channels got zero guidance.

The fix decouples the guidance injection from valid_tool_names: the
model-name gate (TOOL_USE_ENFORCEMENT_MODELS substring check) now
drives whether the block lands, independent of whether the current
session has tools loaded. The valid_tool_names check is preserved
only for:

  - Google-operational sub-block (content is about file paths and
    edit commands — irrelevant without tools)
  - parallel_tool_call_guidance (no point steering a no-tool session
    to batch calls)
  - task_completion_guidance (no point steering a session with no
    work to 'finish')

The model needs the instruction regardless of whether the current
session happens to have tools loaded — a no-tools GLM session today
may turn into a tools session tomorrow, and even within a single
session the model needs to know it should emit structured tool_calls
when those become available rather than falling back to markdown.

This is independent of PR NousResearch#35087 (which widens the OPENAI_ gate to
cover glm/qwen/deepseek via a config-driven tuple) — that PR was
open for 28 days before this issue surfaced. PR NousResearch#35087 addresses
the wrong-family gate; this PR addresses the wrong-gate wrapper.
Both are needed; both can land in any order.

Files:
  agent/system_prompt.py             +49/-22
  tests/agent/test_system_prompt.py  +103/-0

Tests: 14/14 pass in test_system_prompt.py, 170/170 pass in
test_system_prompt.py + test_prompt_builder.py. ruff + ty clean.
9 new regression tests in TestToolUseEnforcementInjectionWithoutTools
cover glm/qwen/deepseek positive, opus negative, parallel-call still
tools-gated, enforce-off respected, explicit-on escape hatch, and
google-operational block still tools-gated.

E2E confirmation: after merge, send a fresh Telegram prompt with
z-ai/glm-5.2 and verify the assistant turn is followed by tool
execution instead of a 1-turn stall.
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists labels Jun 27, 2026

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Approved

Tool-use enforcement guidance injection fix (2 files). Important correctness fix:

  • Removes the if agent.valid_tool_names: guard that was causing zero guidance for gateway sessions starting without tools loaded
  • Makes enforcement model-driven, not tool-driven — correct for models like GLM-5.2 that need the guidance regardless of loaded tools
  • Google-specific guidance still gated on valid_tool_names (correct: it's about how to use tools)
  • Good explanation in the docstring of the root cause (stall after one assistant turn)

This fixes a real regression for Z.AI/GLM models on Telegram/Discord.

@teknium1

Copy link
Copy Markdown
Collaborator

Thanks for isolating the current agent.valid_tool_names gate in agent/system_prompt.py:263; that condition still suppresses enforcement guidance on current main.

Problems

  • The patch changes the no-tools contract but does not update tests/run_agent/test_run_agent.py:1487-1509, which currently asserts the opposite result for tool_use_enforcement=True. The full suite will retain a failing integration test.
  • The moved _inject block also makes GPT/Codex/Grok sessions receive OPENAI_MODEL_EXECUTION_GUIDANCE without tools. That prompt names terminal, execute_code, and web_search (agent/prompt_builder.py:395-414), while the transport omits tools when the schema list is empty (agent/transports/chat_completions.py:554-558).

Suggested changes

  • Update the existing no-tools integration test to the intended contract, rather than adding only a SimpleNamespace regression test.
  • Keep the Google and OpenAI operational blocks tool-gated; make only TOOL_USE_ENFORCEMENT_GUIDANCE model-driven if that is the intended scope.
  • Document or test the gateway tool-resolution path separately: cached gateway agents preserve frozen prompts and tool schemas (gateway/run.py:18155-18157).

This is an automated hermes-sweeper review.

@teknium1 teknium1 added the sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit label Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants