This repository is a PyTorch and PyTorch Lightning implementation of
Perceiver: General Perception with Iterative Attention (paper, video) | ![]() |
Perceiver IO: A General Architecture for Structured Inputs & Outputs (paper, blog post) | ![]() |
General-purpose, long-context autoregressive modeling with Perceiver AR (paper, blog post) | ![]() |
All model classes are written in plain PyTorch and can be wrapped into PyTorch Lightning modules for training at scale. The command line interface is implemented with the Lightning CLI. Pretrained weights can be imported for official models from the 🤗 Hub, training checkpoints from training examples are available for download too. Datasets used in the training examples are 🤗 datasets wrapped into PyTorch Lightning data modules. For NLP tasks, this library supports all 🤗 fast tokenizers and the 🤗 Perceiver UTF-8 bytes tokenizer.
pip install perceiver-io[text,vision]
Installation from sources requires a Miniconda and a Poetry (1.2.0 or higher) installation.
Create and activate the perceiver-io
conda environment:
conda env create -f environment.yml
conda activate perceiver-io
Install main and test dependencies, including all extras:
# Without dependencies required for examples
poetry install --all-extras
If you want to run the examples locally, additionally use --with examples
:
poetry install --all-extras --with examples
docker pull ghcr.io/krasserm/perceiver-io:latest
See Docker image for details.
- Getting started
- Model construction
- Pretrained models
- Training examples
- Inference examples
- Building blocks
Articles referencing this repository: