feat: Gemma 4 day-0 support + token-level OutputRouter (v0.4.0)#58

Merged

raullenchai merged 94 commits intomainfrom

feat/upstream-sync-march26

Apr 6, 2026

Commits on Feb 24, 2026

fix: Use streaming detokenizer for UTF-8-safe incremental decode

janhilgard
and
claude
committed

Commits on Feb 28, 2026

Add --served-model-name CLI parameter
otarkhan
committed
Fix prefix cache dir using served name instead of model path
otarkhan
committed
Add Qwen3.5 model support with text-only loading and fix reasoning+tool streaming

otarkhan
and
claude
committed

Commits on Mar 11, 2026

fix: check trim method existence before calling
OpenClaw
committed

Commits on Mar 12, 2026

Commits on Mar 14, 2026

fix: pass size to ArraysCache in BatchMambaCache for Qwen3.5 hybrid models
neomody77
committed

Commits on Mar 20, 2026

Commits on Mar 21, 2026

Commits on Mar 22, 2026

Commits on Mar 25, 2026

fix: bump mlx-lm minimum to 0.31.0 for hybrid model batching
krystophny
committed

Commits on Mar 26, 2026

merge: sync upstream origin/main — SpecPrefill, native video, MTP injection, served-model-name

Your Name
and
claude
committed
fix: alias validation, Hub model MTP routing, non-streaming text path, use_cache double-pop

Your Name
and
claude
committed
fix: non-streaming text-only MTP deadlock and accumulation bug

Your Name
and
claude
committed
fix: forward stop sequences to text-only MTP generation path

Your Name
and
claude
committed
fix: truncate new_text on stop hit so SSE streams omit stop sequence

Your Name
and
claude
committed

Commits on Mar 27, 2026

fix: use self.max_kv_size instead of None in _make_cache call

Brett Thompson
authored and
Brett Thompson
committed

Commits on Mar 30, 2026

fix: report prompt_tokens correctly for LLM models in SimpleEngine

sjswerdloff
and
clement-7074f29f
committed

Commits on Mar 31, 2026

Commits on Apr 1, 2026

Merge pull request #232 from sjswerdloff/fix/streaming-tool-call-content-leak
Thump604
authored

Commits on Apr 3, 2026

merge: sync upstream origin/main — streaming filters, Anthropic think blocks, detokenizer, v0.2.7

Your Name
and
claude
committed
fix: missing return in load_model_with_fallback success path

Your Name
and
claude
committed
bench: re-run benchmarks post-merge on 4 models

Your Name
and
claude
committed
bench: comprehensive 14-model benchmark + agent integration tests

Your Name
and
claude
committed

Commits on Apr 4, 2026

feat: add Gemma 4 sanitize monkey-patch for mlx-vlm weight loading

Your Name
and
claude
committed
revert: remove Gemma 4 monkey-patch, use mlx-vlm 0.4.3 as-is

Your Name
and
claude
committed
fix: MLLM streaming crash — build_prompt not supported for VLM models

Your Name
and
claude
committed
fix: graceful handling of MLLM stream_chat cleanup errors

Your Name
and
claude
committed

Commits on Apr 5, 2026

deps: bump mlx-vlm minimum to 0.4.4 for Gemma 4 support

Your Name
and
claude
committed
feat: add Gemma 4 tool call parser

Your Name
and
claude
committed
fix: add global exception handler + MLLM error logging for prod resilience

Your Name
and
claude
committed
feat: Gemma 4 text-only LLM path — prompt cache + all optimizations

Your Name
and
claude
committed
bench: Gemma 4 31B — LLM path 5.2x faster than MLLM path

Your Name
and
claude
committed
fix: mixed quantization support for Gemma 4 E4B/E2B models

Your Name
and
claude
committed
fix: strip Gemma 4 thinking tags and turn markers from output

Your Name
and
claude
committed
fix: mixed quant path matching + filter override keys

Your Name
and
claude
committed

Commits on Apr 6, 2026

docs: add Gemma 4 benchmarks to README

Your Name
and
claude
committed
feat: Gemma 4 reasoning parser — streaming thought/content separation

Your Name
and
claude
committed
feat: token-level OutputRouter — config-driven output channel routing

Your Name
and
claude
committed
feat: integrate OutputRouter into streaming pipeline

Your Name
and
claude
committed
fix: 6 issues from self-review on OutputRouter integration

Your Name
and
claude
committed
docs: update Gemma 4 benchmarks — full lineup with OutputRouter

Your Name
and
claude
committed
release: v0.4.0 — Gemma 4 day-0 support + token-level OutputRouter

Your Name
and
claude
committed
fix: suppress orphan tool/response tokens in OutputRouter

Your Name
and
claude
committed
fix: recover text-format tool calls from degraded Gemma 4 output

Your Name
and
claude
committed
feat: final sanitizer — last-mile catch-all against markup leakage

Your Name
and
claude
committed
fix: 3 bugs from self-review on final sanitizer

Your Name
and
claude
committed
bench: final Gemma 4 verification — 0% leak, 6/6 agent tests, all models

Your Name
and
claude
committed
Merge remote-tracking branch 'raullenchai/main' into feat/upstream-sync-march26
Your Name
committed