fix(agent): inject OPENAI_MODEL_EXECUTION_GUIDANCE for GLM/Qwen/DeepSeek - #53847
fix(agent): inject OPENAI_MODEL_EXECUTION_GUIDANCE for GLM/Qwen/DeepSeek#53847tauros1983 wants to merge 1 commit into
Conversation
Telegram sessions on z-ai/glm-5.2 stalled after one assistant turn:
the model replied with a plain-text '[TOOL_CALL]...[/TOOL_CALL]'
marker instead of a structured tool_calls JSON block, so the runtime
saw tool_calls=None and finished with finish_reason=stop. Two
sessions on 2026-06-27 (20260627_144930_7531d0bd, 20260627_145132_
3c5abc52) reproduced this — 3 messages, 0 tool_calls each, then
silence.
Root cause: build_system_prompt_parts only appended
OPENAI_MODEL_EXECUTION_GUIDANCE when the model name contained
'gpt', 'codex', or 'grok'. GLM/Qwen/DeepSeek matched
TOOL_USE_ENFORCEMENT_MODELS (so they got the lighter
TOOL_USE_ENFORCEMENT_GUIDANCE block) but not the execution-discipline
block (tool persistence, mandatory_tool_use, anti-fabrication). On a
weaker tool_use format-trained model that lighter block alone is not
enough to stop the model from emitting its tool intent as natural-
language markdown.
The OPENAI_ prefix reflects origin, not exclusivity (see the comment
on prompt_builder.OPENAI_MODEL_EXECUTION_GUIDANCE). Expand the gate
to the non-Google slice of TOOL_USE_ENFORCEMENT_MODELS — Google keeps
its own more specific GOOGLE_MODEL_OPERATIONAL_GUIDANCE block, so we
skip gemini/gemma here to avoid a duplicate parallel-tool-call steer.
Tests: 8 regression tests in TestOpenAIExecutionGuidanceInjection cover
GLM/DeepSeek/Qwen (positive), GPT/Grok (no regression), Opus/Gemini
(no false-positive), and the off-switch. 169/169 pass in
test_prompt_builder.py + test_system_prompt.py; full suite minus the
unrelated pre-existing test_anthropic_adapter.py MagicMock failure
also clean.
Reproduction evidence (from ~/.hermes/state.db on 2026-06-27):
SELECT message_count, tool_call_count, end_reason FROM sessions
WHERE id IN ('20260627_144930_7531d0bd','20260627_145132_3c5abc52');
-> 3|0|| (both)
SELECT substr(content, 1, 200) FROM messages
WHERE session_id='20260627_144930_7531d0bd' AND role='assistant';
-> 'Ik ga die 3 commits lezen. Eerst kijken waar ik sta...
[TOOL_CALL]\nrun_command\ncommand=cd /Users/guidolassally/esmi-dashboard
2>/dev/null && git log --oneline -1 cc3cd6b92 && ...'
finish_reason was 'stop' and tool_calls was NULL on both sessions.
Live verification (after patch):
python -c 'from unittest.mock import patch; from types import
SimpleNamespace; ... build_system_prompt_parts(agent)["stable"]'
for z-ai/glm-5.2 now contains 'Execution discipline',
'<tool_persistence>', and '<mandatory_tool_use>' sections that
previously were absent.
Related: #35087 solves the same GLM/Qwen/DeepSeek tool-call stall via a different mechanism (new |
tonydwb
left a comment
There was a problem hiding this comment.
Code Review Summary
Verdict: Approved
OpenAI execution guidance injection for GLM/Qwen/DeepSeek (2 files). Important fix:
- Expands the execution guidance injection to cover all
TOOL_USE_ENFORCEMENT_MODELSfamilies (not justgpt/codex/grok) - Uses
TOOL_USE_ENFORCEMENT_MODELSlist directly (maintainable, not hardcoded) - Google models still get the more specific
GOOGLE_MODEL_OPERATIONAL_GUIDANCE(correct) - Comprehensive test coverage: 6 new tests covering GLM, DeepSeek, Qwen, and edge cases
Good fix for a real stall issue. The test suite covers the exact regression scenario.
teknium1
left a comment
There was a problem hiding this comment.
Thanks for isolating a real prompt-tier mismatch. Current main still applies execution discipline only to gpt, codex, and grok at agent/system_prompt.py:289, while the automatic enforcement family tuple includes glm, qwen, and deepseek at agent/prompt_builder.py:302.
Problems
- The new test helper forces
tool_use_enforcement=Trueattests/agent/test_system_prompt.py:120. That bypasses the defaultautoselection path, so the tests do not directly verify theTOOL_USE_ENFORCEMENT_MODELSrelationship the implementation reuses.
Suggested changes
- Add an
auto-mode GLM regression test, plus at least one Qwen or DeepSeek case, asserting both enforcement and execution-discipline guidance are injected. Keep the explicit-off regression test.
This is an automated hermes-sweeper review.
| block (tool persistence, anti-fabrication, mandatory_tool_use). | ||
| """ | ||
|
|
||
| def _prompt(self, model, *, valid_tool_names=("terminal", "read_file"), |
There was a problem hiding this comment.
This default forces the outer enforcement branch, so the new cases do not exercise the production auto route through TOOL_USE_ENFORCEMENT_MODELS. Please add at least one GLM/Qwen/DeepSeek case using tool_use_enforcement="auto" and assert both guidance blocks.
GottZ
left a comment
There was a problem hiding this comment.
This was generated by AI during triage.
Summary
Two PRs address the same one-turn GLM/Qwen/DeepSeek stall through complementary prompt-injection fixes: #53847 extends execution discipline to the affected model families, while #53857 makes baseline tool-use enforcement independent of whether tools are currently loaded.
Related pull requests
- #53847
related— (+85/-4) — primary, needs changes: The diff directly fixes the model-family mismatch by deriving the non-Google execution-guidance families from TOOL_USE_ENFORCEMENT_MODELS, but its tests force enforcement on and therefore miss the automatic-selection path identified by the keep_open contributor review; add auto-mode GLM and Qwen or DeepSeek coverage before merge. - #53857
related— (+149/-27) — complementary but supersedable: The diff fixes the separate no-tools gate that suppresses baseline enforcement in gateway sessions, but it also injects tool-specific OPENAI_MODEL_EXECUTION_GUIDANCE into no-tool GPT/Codex/Grok sessions and leaves the existing integration-test contract unupdated. Despite the keep_open review on #53857, its useful change is the narrower movement of TOOL_USE_ENFORCEMENT_GUIDANCE outside the tools gate, which can be consolidated into #53847 while retaining operational-guidance gating and updating the integration test.
Suggested consolidation
Merge #53847 after adding auto-mode regression coverage and incorporating the narrow no-tools baseline-enforcement fix from #53857, with Google and OpenAI operational guidance remaining tool-gated and the existing no-tools integration test updated. Then close #53857 as superseded by the consolidated #53847 rather than as a duplicate, because the two current diffs fix distinct contributing gates.
Cross-PR triage: Reviewed 2 pull requests and 0 issues in this complex. Each diff was read against this issue; Assessment working set: 16 kB of PR diffs, 12 kB of issue/PR text, 4 kB of discussion (5 comments), 0 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.
Telegram sessions on z-ai/glm-5.2 stalled after one assistant turn: the model replied with a plain-text '[TOOL_CALL]...[/TOOL_CALL]' marker instead of a structured tool_calls JSON block, so the runtime saw tool_calls=None and finished with finish_reason=stop. Two sessions on 2026-06-27 (20260627_144930_7531d0bd, 20260627_145132_ 3c5abc52) reproduced this — 3 messages, 0 tool_calls each, then silence.
Root cause: build_system_prompt_parts only appended OPENAI_MODEL_EXECUTION_GUIDANCE when the model name contained 'gpt', 'codex', or 'grok'. GLM/Qwen/DeepSeek matched TOOL_USE_ENFORCEMENT_MODELS (so they got the lighter TOOL_USE_ENFORCEMENT_GUIDANCE block) but not the execution-discipline block (tool persistence, mandatory_tool_use, anti-fabrication). On a weaker tool_use format-trained model that lighter block alone is not enough to stop the model from emitting its tool intent as natural- language markdown.
The OPENAI_ prefix reflects origin, not exclusivity (see the comment on prompt_builder.OPENAI_MODEL_EXECUTION_GUIDANCE). Expand the gate to the non-Google slice of TOOL_USE_ENFORCEMENT_MODELS — Google keeps its own more specific GOOGLE_MODEL_OPERATIONAL_GUIDANCE block, so we skip gemini/gemma here to avoid a duplicate parallel-tool-call steer.
Tests: 8 regression tests in TestOpenAIExecutionGuidanceInjection cover GLM/DeepSeek/Qwen (positive), GPT/Grok (no regression), Opus/Gemini (no false-positive), and the off-switch. 169/169 pass in test_prompt_builder.py + test_system_prompt.py; full suite minus the unrelated pre-existing test_anthropic_adapter.py MagicMock failure also clean.
Reproduction evidence (from ~/.hermes/state.db on 2026-06-27):
SELECT message_count, tool_call_count, end_reason FROM sessions
WHERE id IN ('20260627_144930_7531d0bd','20260627_145132_3c5abc52');
-> 3|0|| (both)
SELECT substr(content, 1, 200) FROM messages
WHERE session_id='20260627_144930_7531d0bd' AND role='assistant';
-> 'Ik ga die 3 commits lezen. Eerst kijken waar ik sta...
[TOOL_CALL]\nrun_command\ncommand=cd /Users/guidolassally/esmi-dashboard
2>/dev/null && git log --oneline -1 cc3cd6b92 && ...'
finish_reason was 'stop' and tool_calls was NULL on both sessions.
Live verification (after patch):
python -c 'from unittest.mock import patch; from types import
SimpleNamespace; ... build_system_prompt_parts(agent)["stable"]'
for z-ai/glm-5.2 now contains 'Execution discipline',
'<tool_persistence>', and '<mandatory_tool_use>' sections that
previously were absent.
What does this PR do?
Related Issue
Fixes #
Type of Change
Changes Made
How to Test
Checklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests passDocumentation & Housekeeping
docs/, docstrings) — or N/Acli-config.yaml.exampleif I added/changed config keys — or N/ACONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — or N/AFor New Skills
hermes --toolsets skills -q "Use the X skill to do Y"Screenshots / Logs