Phi4 mini

[ysjprojects]

New Phi 4 Mini Instruct 3.8B language model.

See issue #1948

• Outperforms larger models in text-based tasks, including reasoning, math, coding, instruction-following, and function-calling. (e.g. beats Qwen2.5-7B-Instruct in HumanEval)
• Supports 128k context size.

[Borda]

@ysjprojects mind have look at the failing tests:

FAILED tests/convert/test_hf_checkpoint.py::test_convert_hf_checkpoint[phi-2] - KeyError: 'transformer.wte.weight'
FAILED tests/test_model.py::test_against_hf_phi_3[device0-dtype0-Phi-4-mini-instruct] - AssertionError: Tensor-likes are not close!

[ysjprojects]

@ysjprojects mind have look at the failing tests:

FAILED tests/convert/test_hf_checkpoint.py::test_convert_hf_checkpoint[phi-2] - KeyError: 'transformer.wte.weight'
FAILED tests/test_model.py::test_against_hf_phi_3[device0-dtype0-Phi-4-mini-instruct] - AssertionError: Tensor-likes are not close!

Hey @Borda, the first test was fixed.

For the second test, it runs successfully but only when transformers is upgraded to 4.49.0

It's a specific issue with phi-4-mini-instruct that is backward incompatible with previous versions. I have also raised this issue here: #1968

[lantiga]

Looking good, but we're being rate limited by HF on CI tests test_tokenizer_against_hf (on #1952, #1897 as well)

We should cache those assets on our end /cc @Borda