Skip to content

ggml : add CPU backend reference implementation (wip)#16004

Open
danbev wants to merge 1 commit intoggml-org:masterfrom
danbev:test-backend-ops-all-cpu-variants
Open

ggml : add CPU backend reference implementation (wip)#16004
danbev wants to merge 1 commit intoggml-org:masterfrom
danbev:test-backend-ops-all-cpu-variants

Conversation

@danbev
Copy link
Member

@danbev danbev commented Sep 15, 2025

This commit introduces a CPU reference implementation for GGML,
designed primarily for testing and validation purposes.

The motivation for this addition is to have a pure C CPU backend
implementation that does not use any hardware-specific optimizations
or intrinsics. This will allow for testing the CPU backend variants
against the reference implementation to ensure correctness


Building

$ cmake -B build \
    -DGGML_CPU_REF_BACKEND=ON
    -DGGML_BACKEND_DL=ON \
    -DGGML_CPU_ALL_VARIANTS=ON

List availble cpu architectures/variants:

$ ./build/bin/test-backend-ops cpu-variants --list
CPU variants:
  CPU-haswell     - 12th Gen Intel(R) Core(TM) i7-1260P
  CPU-sse42       - 12th Gen Intel(R) Core(TM) i7-1260P
  CPU-x64         - 12th Gen Intel(R) Core(TM) i7-1260P
  CPU-alderlake   - 12th Gen Intel(R) Core(TM) i7-1260P
  CPU-sandybridge - 12th Gen Intel(R) Core(TM) i7-1260P

Run tests:

./build-ref/bin/test-backend-ops cpu-variants --variant CPU-alderlake -o ADD
CPU-ref features:
  SSE2 = 1
CPU-alderlake features:
  SSE2 = 1
  SSE3 = 1
  SSSE3 = 1
  AVX = 1
  AVX_VNNI = 1
  AVX2 = 1
  F16C = 1
  FMA = 1
  BMI2 = 1
  LLAMAFILE = 1
  OPENMP = 1
  REPACK = 1
Testing CPU variant 'CPU-alderlake' against 'CPU-ref' backend...

 ADD(type=f16,ne=[1,1,8,1],nr=[1,1,1,1],nf=1): OK
 ADD(type=f16,ne=[1,1,1,1],nr=[32,1,1,1],nf=1): OK
 ...

@slaren
Copy link
Member

slaren commented Sep 15, 2025

There is no need to duplicate the CPU backend to do this - that would be a maintenance nightmare. You can add variant that is built in similar way that all the other CPU backend variants when using GGML_CPU_ALL_VARIANTS, just with all the instruction sets disabled so that only the C implementation is used.

@github-actions github-actions bot added testing Everything test related ggml changes relating to the ggml tensor library for machine learning labels Sep 15, 2025
@danbev
Copy link
Member Author

danbev commented Sep 15, 2025

@slaren Ah right, that does sound much better and simpler than this. I'll make the changes. Thanks!

@danbev danbev force-pushed the test-backend-ops-all-cpu-variants branch 2 times, most recently from c9806e0 to a509662 Compare September 16, 2025 04:05
@danbev danbev marked this pull request as ready for review September 17, 2025 08:10
@danbev danbev force-pushed the test-backend-ops-all-cpu-variants branch from 9601e7a to 3b93a39 Compare November 5, 2025 06:14
This commit introduces a CPU reference implementation for GGML,
designed primarily for testing and validation purposes.

The motivation for this addition is to have a pure C CPU backend
implementation that does not use any hardware-specific optimizations
or intrinsics. This will allow for testing the CPU backend variants
against the reference implementation to ensure correctness

Building:
```console
$ cmake -B build \
    -DGGML_CPU_REF_BACKEND=ON
    -DGGML_BACKEND_DL=ON \
    -DGGML_CPU_ALL_VARIANTS=ON
```

List availble cpu architectures/variants:
```console
$ ./build/bin/test-backend-ops cpu-variants --list
CPU variants:
  CPU-haswell     - 12th Gen Intel(R) Core(TM) i7-1260P
  CPU-sse42       - 12th Gen Intel(R) Core(TM) i7-1260P
  CPU-x64         - 12th Gen Intel(R) Core(TM) i7-1260P
  CPU-alderlake   - 12th Gen Intel(R) Core(TM) i7-1260P
  CPU-sandybridge - 12th Gen Intel(R) Core(TM) i7-1260P
```
Run tests:
```console
./build-ref/bin/test-backend-ops cpu-variants --variant CPU-alderlake -o ADD
CPU-ref features:
  SSE2 = 1
CPU-alderlake features:
  SSE2 = 1
  SSE3 = 1
  SSSE3 = 1
  AVX = 1
  AVX_VNNI = 1
  AVX2 = 1
  F16C = 1
  FMA = 1
  BMI2 = 1
  LLAMAFILE = 1
  OPENMP = 1
  REPACK = 1
Testing CPU variant 'CPU-alderlake' against 'CPU-ref' backend...

 ADD(type=f16,ne=[1,1,8,1],nr=[1,1,1,1],nf=1): OK
 ADD(type=f16,ne=[1,1,1,1],nr=[32,1,1,1],nf=1): OK
 ...
```
@danbev danbev force-pushed the test-backend-ops-all-cpu-variants branch from 3b93a39 to 2f7d0ac Compare January 2, 2026 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ggml changes relating to the ggml tensor library for machine learning testing Everything test related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants