Skip to content

resloved/RWKV-notebooks

Repository files navigation

RWKV Notebooks

Notebooks related to RWKV – a Parallelizable RNN

Inference:

A demo of ChatRWKV

A demo for using the LangChain wrapper with RWKV

Fine-tuning:

The goal for these notebooks is to simplify fine-tuning of the RWKV models trained by BlinkDL on the Pile

With small changes you should be able to train/tune any RWKV model

Only necessary for fine-tuning legacy models

Edits training files to attempt to stay up to date to changes to the RWKV repo, may cause unexpected instability

Most stable option using the latest command line interface for training

Uses the LoRA fork of RWKV by Blealtan

Requires less VRAM and is recommended for most fine-tuning tasks

About

πŸ“– β€” Notebooks related to RWKV

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published