Add memory-efficient chunked conversion for large models

[iamadalek]

Summary

• Adds --chunked flag to mlx_lm.convert that processes weights one transformer layer at a time, enabling conversion of models larger than available RAM
• New mlx_lm/chunked_convert.py module with streaming pipeline: safetensors header scanning, layer-based chunk planning, quantization plan building, per-chunk load/sanitize/cast/quantize/save with aggressive memory reclamation between chunks
• Produces output identical to the standard conversion path (verified by bit-for-bit comparison tests on synthetic models)
• Supports standard quantization (affine, mxfp4, nvfp4, mxfp8), mixed recipes, and dtype-only conversion
• Incompatible combinations (--dequantize, AWQ/GPTQ/DWQ, VL models, custom model_file) raise clear errors; single-file models fall back to standard conversion automatically

Benchmark

Synthetic 8-layer Llama model (hidden=1024, vocab=32000, 642 MB total):

Mode	Peak Memory	Ratio
Standard	742 MB	1x
Chunked	164 MB	4.5x reduction

The ratio scales with layer count. An 8-layer model processes ~1 layer at a time (plus embeddings/lm_head). A 32-layer model would see ~16x reduction. For the target use case (DeepSeek V3 at 671B), peak memory drops from ~370GB (full model) to ~29GB (one MoE layer).

Test plan

• Unit tests: safetensors header parsing, layer chunk planning, quantized size estimation, quant plan building, validation checks, chunked sanitize
• Integration test: bit-for-bit comparison of chunked vs standard conversion on synthetic Llama model (4-layer, quantized)
• Integration test: MoE expert stacking via chunked sanitize on synthetic Mixtral model (sentinel injection for non-zero layers)
• Integration test: dtype-only chunked conversion
• Error case: --chunked + --dequantize raises ValueError
• CLI tests: --chunked flag registered in parser with correct default
• Full test suite: all 176 tests pass (175 existing + 1 new)
• Linting: black + isort clean

Closes #876

🤖 Generated with Claude Code

[awni]

This is not necessary.. it's already possible to convert a model larger than the RAM of the machine as the conversion is streamed in chunks already.

[iamadalek]

This is not necessary.. it's already possible to convert a model larger than the RAM of the machine as the conversion is streamed in chunks already.

Thanks for the quick review, Awni. I appreciate the clarification — I should have dug deeper into how MLX’s lazy evaluation handles the memory pressure during conversion before assuming the bottleneck was where I thought it was. Good to know the existing path already streams effectively.