fix: route text-only requests through MLLM scheduler for vision models#52

Closed

janhilgard wants to merge 32 commits intowaybarrios:mainfrom

janhilgard:fix/mllm-text-only-crash

Commits on Feb 6, 2026

feat: Add Anthropic Messages API endpoint with tool call parser fallback

janhilgard
and
claude
committed
Fix VRAM spike at end of generation via lazy-eval-safe cache handling

janhilgard
and
claude
committed
Add mid-prefill cache saving, disconnect detection, and cancellation safety

janhilgard
and
claude
committed
Fix lint: rename ambiguous variable `l`, remove unused import

janhilgard
and
claude
committed
Fix mid-prefill cache restore: convert BatchKVCache to KVCache

janhilgard
and
claude
committed
Fix cache fetch: prefer supersequence match over prefix match

janhilgard
and
claude
committed
Add prefix cache sharing for requests with common prefix

janhilgard
and
claude
committed
Fix Metal SIGABRT crash when aborting chunked prefill mid-stream

janhilgard
and
claude
committed

Commits on Feb 7, 2026

Commits on Feb 8, 2026

Commits on Feb 9, 2026

Commits on Feb 10, 2026

Merge remote-tracking branch 'origin/main' into fix/mllm-text-only-crash
janhilgard
committed