Skip to content

fix(agent): enable tool_stream for zai chat transport - #14619

Open
Srnki wants to merge 1 commit into
NousResearch:mainfrom
Srnki:fix/zai-tool-stream-chat-transport
Open

fix(agent): enable tool_stream for zai chat transport#14619
Srnki wants to merge 1 commit into
NousResearch:mainfrom
Srnki:fix/zai-tool-stream-chat-transport

Conversation

@Srnki

@Srnki Srnki commented Apr 23, 2026

Copy link
Copy Markdown

What does this PR do?

This PR enables tool_stream=true for Z.AI / BigModel chat-completions requests in the current transport-based code path.

GLM models on Z.AI can buffer large tool-call arguments and remain silent long enough to hit the provider's idle timeout. Hermes already has an open fix for the older _build_api_kwargs path, but the current codebase now assembles extra_body inside agent/transports/chat_completions.py.

This change ports the same behavior to the current transport path by detecting Z.AI endpoints from provider / base_url and enabling tool_stream only when tools are present.

Related Issue

Fixes #

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • Passed provider and base_url from run_agent.py into ChatCompletionsTransport.build_kwargs()
  • Updated agent/transports/chat_completions.py to set extra_body["tool_stream"] = True when:
    • provider == "zai", or
    • base_url contains z.ai or bigmodel.cn
  • Limited the behavior to requests that include tools
  • Used setdefault() so explicit overrides still win
  • Added regression coverage in tests/agent/transports/test_chat_completions.py for:
    • Z.AI tool-call requests enabling tool_stream
    • non-Z.AI providers not enabling it

How to Test

  1. Run:
    scripts/run_tests.sh tests/agent/transports/test_chat_completions.py -v
  2. Run Hermes against Z.AI with tool calls, for example:
    HERMES_DUMP_REQUESTS=1 OPENAI_BASE_URL='https://api.z.ai/api/coding/paas/v4' hermes chat -q 'Use the terminal tool to run: printf TOOL_OK' -m glm-5 --provider zai -Q --yolo
  3. Verify Hermes successfully executes the tool call and confirm the dumped request payload includes:
    "extra_body": {
      "tool_stream": true
    }

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: macOS

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Screenshots / Logs

Manual validation against Z.AI succeeded with a real Hermes tool-call run:

  • endpoint hit: https://api.z.ai/api/coding/paas/v4/chat/completions
  • tool call completed successfully with output: TOOL_OK
  • request dumps confirmed extra_body.tool_stream = true

There is already an open PR for the older code path: #12758. This PR applies the same fix to the current transport-based implementation.

Note: the full repository test suite currently has unrelated pre-existing failures, so this PR adds targeted regression coverage for the transport path changed here.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/zai ZAI provider labels Apr 23, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for isolating the Z.AI tool-streaming behavior. The automatic default is still absent on current main, but this patch now lands on a bypassed path.

Problems

  • Registered Z.AI requests resolve through agent/chat_completion_helpers.py:849-888 with provider_profile; agent/transports/chat_completions.py:316-321 returns through _build_kwargs_from_profile before the legacy extra_body block changed here. plugins/model-providers/zai/__init__.py:88-108 currently has no tool_stream emission.
  • run_agent.py:5328-5331 is now a forwarder after extraction commit 4b25619bc4770396faf206429ddc180ad02231a9; the request-builder body is in agent/chat_completion_helpers.py.
  • The new test omits provider_profile, so it validates only the legacy fallback rather than the direct Z.AI runtime route.

Suggested changes

  • Move the tools-only Z.AI/base-URL gate into the current profile-aware assembly, covering both zai and custom Z.AI endpoints.
  • Preserve override precedence by applying the default before request_overrides.extra_body merges at agent/transports/chat_completions.py:601-608, then add profile-path regression coverage.

Automated hermes-sweeper review.

)

if _is_zai_endpoint and tools:
extra_body.setdefault("tool_stream", True)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This branch is now below the provider_profile early return on current main (agent/transports/chat_completions.py:316-321), so direct provider="zai" requests never reach it. Please place the default in the profile-aware assembly as well, while retaining request-override precedence.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists provider/zai ZAI provider sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants