How to enable simd in candle

[corysabol]

I'm working on a project in which I am focusing on CPU inference. I was wondering how I can enable simd for the candle crate. Is this something I would need to do by building from source, or is there some other method in which I can pass the rustflags through my cargo.toml to the candle-core crate?

For example when I compile the mistral example and then run it, it shows that I do not have simd128 enabled

./target/release/examples/mistral --prompt "Here is a sample quick sort implementation in rust " --quantized -n 400
avx: true, neon: false, simd128: false, f16c: true

cat /proc/cpuinfo | grep -e "avx" -e "sse4_2" shows that my CPU supports both of those features (I think at least I could be interpreting this incorrectly).

Some obligatory system info

Linux pop-os 6.6.10-76060610-generic #202401051437~1704728131~22.04~24d69e2 SMP PREEMPT_DYNAMIC Mon J x86_64 x86_64 x86_64 GNU/Linux

64GB RAM

Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         43 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  24
  On-line CPU(s) list:   0-23
Vendor ID:               AuthenticAMD
  Model name:            AMD Ryzen 9 3900X 12-Core Processor
    CPU family:          23
    Model:               113
    Thread(s) per core:  2
    Core(s) per socket:  12
    Socket(s):           1
    Stepping:            0
    Frequency boost:     enabled
    CPU max MHz:         4672.0698
    CPU min MHz:         2200.0000
    BogoMIPS:            7585.73
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mm
                         xext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmu
                         lqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr
                         8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr
                         _llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed 
                         adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero i
                         rperf xsaveerptr rdpru wbnoinvd arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pa
                         usefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip rdpid overflow_recov succor smca sev sev_es

[LaurentMazare]

simd128 is the name for wasm simd so it's expected not to have it on your platform. The avx: true in the output shows that simd support is actually enabled.

[corysabol]

Ahh okay, thanks for the swift answer. I think that settles it then. I'm dipping my toes into this stuff and learning as I go :)

Cheers!