llama.cpp backend: GPU offload and sequence-copy fan-out for shared mode - #34

llama.cpp backend: GPU offload and sequence-copy fan-out for shared mode#34

maximelefrancois86 wants to merge 7 commits into

TheoLeeCJ:masterTheoLeeCJ/SemIf-OpenJev:masterfrom

maximelefrancois86:llamacpp-cuda-parallelmaximelefrancois86/SemIf:llamacpp-cuda-parallelCopy head branch name to clipboard