Skip to content

Conversation

@coffeerunhobby
Copy link
Contributor

Description

Fix: Prevent BMI2 instruction crash on AVX-only CPUs
Problem
The llama-cpp-avx binary selection incorrectly runs on CPUs with AVX but without BMI2
(e.g., Intel Ivy Bridge). This causes crashes with "EOF" errors during model warmup
because the binary contains BMI2 instructions (shlx, mulx, etc.).

Notes for Reviewers

Solution:
Add BMI2 feature check before selecting the AVX binary. CPUs with AVX but without BMI2
now safely fall back to llama-cpp-fallback:

llama-cpp-fallback - x86_64 (SSE2 only)
llama-cpp-avx - AVX + no BMI2 (-mno-bmi -mno-bmi2)
llama-cpp-avx2 - AVX2 + FMA + BMI2 (unchanged)
llama-cpp-avx512 - AVX-512 (unchanged)

Testing:

  • Intel Xeon E3-1240 V2 (AVX, no BMI2): now uses fallback
  • Intel Haswell+ (AVX2 + BMI2): behavior unchanged

Affected CPUs

  • Intel Sandy Bridge / Ivy Bridge (2011-2013)
  • AMD Bulldozer / Piledriver (2011-2014)

Signed-off-by: Coffee Run Hobby 294crhwtf@mozmail.com

@netlify
Copy link

netlify bot commented Jan 1, 2026

Deploy Preview for localai ready!

Name Link
🔨 Latest commit bd3ccbd
🔍 Latest deploy log https://app.netlify.com/projects/localai/deploys/69583910b99e71000716ec14
😎 Deploy Preview https://deploy-preview-7817--localai.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@mudler
Copy link
Owner

mudler commented Jan 2, 2026

Good catch, this looks good, however it fails on apple - would make sense to ignore the option when building on darwin

@mudler mudler force-pushed the fix-missing-bmi2-avx-cpus branch from d4d9209 to 0f7f87d Compare January 2, 2026 20:47
@github-actions github-actions bot added kind/documentation Improvements or additions to documentation area/ai-model dependencies labels Jan 2, 2026
@mudler mudler force-pushed the fix-missing-bmi2-avx-cpus branch from 0f7f87d to 851ec30 Compare January 2, 2026 20:47
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler mudler force-pushed the fix-missing-bmi2-avx-cpus branch from 851ec30 to 9b7cc4b Compare January 2, 2026 20:49
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler mudler force-pushed the fix-missing-bmi2-avx-cpus branch from 9b7cc4b to bd3ccbd Compare January 2, 2026 21:30
@mudler mudler merged commit 666d110 into mudler:master Jan 3, 2026
38 of 39 checks passed
@mudler mudler added bug Something isn't working and removed kind/documentation Improvements or additions to documentation area/ai-model dependencies labels Jan 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants