Improve `cvector-generator`

[jakexcosme]

Note: This issue was copied from ggml-org#8724

Original Author: @ngxson
Original Issue Number: ggml-org#8724
Created: 2024-07-27T13:05:20Z

---

• Fix random direction of vector produced by PCA method cvector: better prompt handling, add "mean vector" method ggml-org/llama.cpp#8069 (comment) (cc @jukofyork)
• Performance: Using batch and multi-sequences (for llama_decode)
• Performance: Allow using GPU (need to test, because some backends don't support certain ops)
• Add SVD and UMAP
• Add tests if possible

Ref: ggml-org#7514

Note: I'm writing this for tracking, but people are free to do PRs if you have any ideas to share. I'm planning to do some of the tasks in August 2024.