Skip to content

fix(agent): preload jiter native parser - #33692

Merged
teknium1 merged 1 commit into
mainfrom
hermes/hermes-ce53310b
May 28, 2026
Merged

fix(agent): preload jiter native parser#33692
teknium1 merged 1 commit into
mainfrom
hermes/hermes-ce53310b

Conversation

@teknium1

@teknium1 teknium1 commented May 28, 2026

Copy link
Copy Markdown
Contributor

Salvage of #33665 by @helix4u onto current main.

Summary

Preloads the OpenAI SDK's native jiter streaming parser during agent package import, so the first DeepSeek/OpenAI-compatible streaming request on Windows doesn't hit ModuleNotFoundError: No module named 'jiter.jiter'.

Root cause

OpenAI SDK lazy-imports openai.lib.streaming.chat._completions the first time a streaming response is consumed, from inside the SSE worker thread. That module does from jiter import from_json, which triggers jiter/__init__.pyfrom .jiter import * → load of the native .pyd. On some Windows installs this thread-load fails (likely an AV / file-lock / loader-state race), even though the same import jiter.jiter succeeds when run synchronously from the Hermes venv at a REPL.

Reproduced live by a Discord user (Lucien) on Windows + DeepSeek: direct import jiter.jiter printed jiter ok, but every streaming chat request failed with ModuleNotFoundError: No module named 'jiter.jiter'. Confirmed workaround:

.\venv\Scripts\python.exe -c "import jiter.jiter, runpy; runpy.run_module('hermes_cli.main', run_name='__main__')"

That preloads the native extension on the main thread before any Hermes code starts. The PR makes Hermes do the same automatically.

Changes

  • agent/jiter_preload.py: best-effort importlib.import_module("jiter.jiter") + from jiter import from_json, runs at module import. Swallows exceptions so a truly missing/broken install still surfaces via the normal SDK error path.
  • agent/__init__.py: imports jiter_preload so the side effect runs on first import agent.* (which happens very early in CLI/gateway startup via agent.portal_tags, agent.secret_sources, etc.).
  • tests/agent/test_jiter_preload.py: happy path + ModuleNotFoundError fallback.

Validation

  • scripts/run_tests.sh tests/agent/test_jiter_preload.py → 2/2 passed.
  • Local python3 -c "import agent; from agent import jiter_preload; print(jiter_preload._JITER_PRELOADED)"True, jiter.jiter in sys.modules.
  • Author email already mapped in scripts/release.py.

Closes #33665. helix4u authorship preserved via rebase-merge.

Infographic

preload-jiter-native-parser

@github-actions

Copy link
Copy Markdown
Contributor

🔎 Lint report: hermes/hermes-ce53310b vs origin/main

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 9522 on HEAD, 9521 on base (🆕 +1)

🆕 New issues (1):

Rule Count
unresolved-import 1
First entries
agent/jiter_preload.py:29: [unresolved-import] unresolved-import: Cannot resolve imported module `jiter`

✅ Fixed issues: none

Unchanged: 5017 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels May 28, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Salvage of #33665 by @helix4u onto current main. Both fix the same jiter preload issue on Windows — this PR preserves the original approach with maintainer authorship.

@teknium1
teknium1 merged commit 459d769 into main May 28, 2026
26 of 28 checks passed
@teknium1
teknium1 deleted the hermes/hermes-ce53310b branch May 28, 2026 07:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants