Skip to content

A PyTorch implementation of Perceiver, Perceiver IO and Perceiver AR with PyTorch Lightning scripts for distributed training

License

Notifications You must be signed in to change notification settings

krasserm/perceiver-io

Folders and files

NameName
Last commit message
Last commit date

Latest commit

19ba30a Â· Mar 31, 2023
Jan 27, 2023
Mar 31, 2023
Mar 31, 2023
Feb 27, 2023
Feb 25, 2023
Aug 21, 2022
Feb 7, 2022
Nov 20, 2022
Feb 7, 2022
Mar 31, 2023
Feb 24, 2023
Feb 21, 2023
Oct 1, 2021
Mar 31, 2023
Feb 21, 2023
Feb 21, 2023
Mar 31, 2023
Nov 20, 2022

Repository files navigation

Perceiver, Perceiver IO and Perceiver AR

This repository is a PyTorch and PyTorch Lightning implementation of

Perceiver: General Perception with Iterative Attention (paper, video) Perceiver
Perceiver IO: A General Architecture for Structured Inputs & Outputs (paper, blog post) Perceiver IO
General-purpose, long-context autoregressive modeling with Perceiver AR (paper, blog post) Perceiver AR

All model classes are written in plain PyTorch and can be wrapped into PyTorch Lightning modules for training at scale. The command line interface is implemented with the Lightning CLI. Pretrained weights can be imported for official models from the 🤗 Hub, training checkpoints from training examples are available for download too. Datasets used in the training examples are 🤗 datasets wrapped into PyTorch Lightning data modules. For NLP tasks, this library supports all 🤗 fast tokenizers and the 🤗 Perceiver UTF-8 bytes tokenizer.

Installation

Via pip

pip install perceiver-io[text,vision]

From sources

Installation from sources requires a Miniconda and a Poetry (1.2.0 or higher) installation.

Create and activate the perceiver-io conda environment:

conda env create -f environment.yml
conda activate perceiver-io

Install main and test dependencies, including all extras:

# Without dependencies required for examples
poetry install --all-extras

If you want to run the examples locally, additionally use --with examples:

poetry install --all-extras --with examples

Docker image

docker pull ghcr.io/krasserm/perceiver-io:latest

See Docker image for details.

Documentation

Articles

Articles referencing this repository:

Other implementations