ggml : add CPU backend reference implementation (wip)#16004
Open
danbev wants to merge 1 commit intoggml-org:masterfrom
Open
ggml : add CPU backend reference implementation (wip)#16004danbev wants to merge 1 commit intoggml-org:masterfrom
danbev wants to merge 1 commit intoggml-org:masterfrom
Conversation
Member
|
There is no need to duplicate the CPU backend to do this - that would be a maintenance nightmare. You can add variant that is built in similar way that all the other CPU backend variants when using |
Member
Author
|
@slaren Ah right, that does sound much better and simpler than this. I'll make the changes. Thanks! |
c9806e0 to
a509662
Compare
9601e7a to
3b93a39
Compare
This commit introduces a CPU reference implementation for GGML,
designed primarily for testing and validation purposes.
The motivation for this addition is to have a pure C CPU backend
implementation that does not use any hardware-specific optimizations
or intrinsics. This will allow for testing the CPU backend variants
against the reference implementation to ensure correctness
Building:
```console
$ cmake -B build \
-DGGML_CPU_REF_BACKEND=ON
-DGGML_BACKEND_DL=ON \
-DGGML_CPU_ALL_VARIANTS=ON
```
List availble cpu architectures/variants:
```console
$ ./build/bin/test-backend-ops cpu-variants --list
CPU variants:
CPU-haswell - 12th Gen Intel(R) Core(TM) i7-1260P
CPU-sse42 - 12th Gen Intel(R) Core(TM) i7-1260P
CPU-x64 - 12th Gen Intel(R) Core(TM) i7-1260P
CPU-alderlake - 12th Gen Intel(R) Core(TM) i7-1260P
CPU-sandybridge - 12th Gen Intel(R) Core(TM) i7-1260P
```
Run tests:
```console
./build-ref/bin/test-backend-ops cpu-variants --variant CPU-alderlake -o ADD
CPU-ref features:
SSE2 = 1
CPU-alderlake features:
SSE2 = 1
SSE3 = 1
SSSE3 = 1
AVX = 1
AVX_VNNI = 1
AVX2 = 1
F16C = 1
FMA = 1
BMI2 = 1
LLAMAFILE = 1
OPENMP = 1
REPACK = 1
Testing CPU variant 'CPU-alderlake' against 'CPU-ref' backend...
ADD(type=f16,ne=[1,1,8,1],nr=[1,1,1,1],nf=1): OK
ADD(type=f16,ne=[1,1,1,1],nr=[32,1,1,1],nf=1): OK
...
```
3b93a39 to
2f7d0ac
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit introduces a CPU reference implementation for GGML,
designed primarily for testing and validation purposes.
The motivation for this addition is to have a pure C CPU backend
implementation that does not use any hardware-specific optimizations
or intrinsics. This will allow for testing the CPU backend variants
against the reference implementation to ensure correctness
Building
List availble cpu architectures/variants:
Run tests: