Skip to content

mrsteyk/rwkvk-rs

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

33 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

RWKV, but in Rust

This is pretty much your usual RWKV, but it follows the RIIR principle.

Notable tech stack uses are memmap2 and safetensors.

FAQ

Are there other projects?

Yes!

How to install?

This can break and if it does use --rev or --tag flags to specify which version to install.

cargo +nightly install --git https://github.com/mrsteyk/rwkvk-rs --features="build-binary"

How to run?

Example:

cargo run --features build-binary --release -- -t ../../RWKV-LM-deepspeed/20B_tokenizer_openchatgpt.json -m ../../RWKV-LM-deepspeed/RWKV-v4neo/RWKV-4-Pile-430M-20220808-8066.rnn.bf16.safetensors "hi!"

How do I get a model for this?

Either download one from hf:mrsteyk/RWKV-LM-safetensors or convert yourself using convert_safetensors.py from gh:mrsteyk/RWKV-LM-deepspeed with arguments --bf16 --rnn. Make sure to read the next section about alignment.

I get some weird errors when I try to use models I converted myself?

UPDATE

Update your safetensors version! gh:huggingface/safetensors#148 got merged!

OLD (kept for historic purposes)

See gh:huggingface/safetensors#178.

TL;DR you need to pad the header, no need to reorder tensors because I copy time_ tensors (for now at least).

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages