feat: MiniMax-M2 support, Tier 0 optimizations & pinned prefix cache

[raullenchai]

Summary

MiniMax-M2.5 model support with full tool calling, speculative decoding, configurable GPU memory management, streaming error resilience, Tier 0 performance optimizations, and a comprehensive benchmark suite.

MiniMax-M2 Support

• Tool call parser — Full XML-based tool call parsing (<minimax:tool_call>/<invoke> format) with streaming support, reasoning parser integration, and hallucinated tag filtering
• Speculative decoding — Draft model support via --draft-model for 1.2-1.4x decode speedup, with HybridEngine for shared model state

Infrastructure

• Configurable GPU memory — --gpu-memory-utilization flag to control Metal soft allocation limit and emergency cache clear threshold (fixes 3.5x slowdown on 200GB+ models)
• Streaming error resilience — _disconnect_guard() now catches all exceptions, logs server-side, sends SSE error event to client

Tier 0 Optimizations

• GC control — Disable Python GC during generation to eliminate latency spikes with large models (120GB+). --gc-control (default: enabled), --no-gc-control to disable
• Pinned prefix cache — Prevent system prompt eviction under memory pressure. is_pinned flag on CacheBlock, skip-pinned eviction in LRU queue, pin_blocks()/unpin_blocks() API, pin_prefix()/unpin_prefix() on both PrefixCacheManager and BlockAwarePrefixCache. --pin-system-prompt auto-pins after first request
• Schema error hardening — Guided generation falls back to standard generation on failure instead of 500 error
• Optimization roadmap — Tier 0-2 roadmap added to README with upstream vLLM PR references

Benchmark Suite

• Six-dimension benchmark for MiniMax-M2.5 on M3 Ultra 256GB with baseline results

Commits

1. feat: Add MiniMax-M2 tool call parser with streaming support
2. feat: Add --gpu-memory-utilization for configurable memory limits
3. feat: Add speculative decoding support with draft models
4. fix: Handle unexpected exceptions in streaming disconnect guard
5. feat: Add comprehensive benchmark suite with baseline results
6. feat: Add Tier 0 optimizations and pinned prefix cache

Key files changed

File	Description
vllm_mlx/tool_parsers/minimax_tool_parser.py	MiniMax XML tool call parser with streaming
vllm_mlx/engine/hybrid.py	HybridEngine for speculative + batched mode
vllm_mlx/speculative/prompt_lookup.py	Prompt lookup speculative decoding
vllm_mlx/api/guided.py	JSON schema enforcement with outlines + schema error hardening
vllm_mlx/server.py	GC control, guided gen fallback, auto-pin system prompt, streaming tool calls, _disconnect_guard hardening
vllm_mlx/cli.py	--gc-control, --pin-system-prompt, --draft-model, --gpu-memory-utilization, minimax parser
vllm_mlx/paged_cache.py	is_pinned on CacheBlock, skip-pinned eviction, pin_blocks()/unpin_blocks() API
vllm_mlx/prefix_cache.py	pin_prefix()/unpin_prefix() on PrefixCacheManager and BlockAwarePrefixCache
benchmark_minmax.py	Comprehensive benchmark script
docs/plans/tier0-pinned-prefix-cache.md	Tier 0 implementation plan
README.md	Baseline benchmark + Tier 0-2 optimization roadmap

Benchmark Results (MiniMax-M2.5, M3 Ultra 256GB)

Metric	Result
TTFT (short prompt)	0.33s
TTFT (long prompt)	1.40s
Decode speed	49-54 tok/s
Prefix cache Turn4/Turn1 ratio	2.09x
Tool calling accuracy	4/4 (100%)
Long generation (8192 tok)	Stable, no crash

Test plan

• Paged cache tests (37 passed)
• Prefix cache tests (21 passed)
• API model tests (229 passed)
• MiniMax tool calling: simple, multi-arg, code execution, multi-tool
• Streaming with reasoning parser + tool call detection
• _disconnect_guard error handling
• --gpu-memory-utilization at 0.90 and 0.95 on 200GB+ model

🤖 Generated with Claude Code