Skip to content

simple-engine: keep tool chat on the streaming execution path#285

Merged
Thump604 merged 5 commits intowaybarrios:mainfrom
Thump604:codex/simpleengine-tool-chat-restack
Apr 11, 2026
Merged

simple-engine: keep tool chat on the streaming execution path#285
Thump604 merged 5 commits intowaybarrios:mainfrom
Thump604:codex/simpleengine-tool-chat-restack

Conversation

@Thump604
Copy link
Copy Markdown
Collaborator

Supersedes #222 with the same functional change rebased on current main.

Why this PR exists:

  • I rebased the original branch locally and validated it cleanly.
  • Direct maintainer push back to the original computor-org fork is not available from this auth context, so the original PR could not be updated in place.

Scope:

  • route tool-enabled non-stream SimpleEngine.chat() through the streaming chat path
  • preserve streamed token ids and final finish_reason/completion accounting
  • carry the regression coverage in tests/test_simple_engine.py on top of current main

Local validation:

  • python -m py_compile vllm_mlx/engine/simple.py tests/test_simple_engine.py
  • python -m black --check --fast vllm_mlx/engine/simple.py tests/test_simple_engine.py
  • pytest -q tests/test_simple_engine.py

Authorship note: this restack preserves the original implementation from #222; the only local adjustment was aligning the regression assertion with current main's cleaned tool-call text behavior.

krystophny and others added 5 commits April 11, 2026 11:29
* fix: unify tool-enabled simple chat on streaming path

* fix: preserve simple chat contracts on streaming path

* fix: keep tool chat on the streaming execution path

* fix: preserve streamed completion token counts
The try/except block computing `tokens` via tokenizer.encode() was
unused -- the return statement already reads from final_output.tokens.
@Thump604 Thump604 merged commit 6bb4142 into waybarrios:main Apr 11, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants