Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge CalBERT-rewrite with Main #1

Merged
merged 43 commits into from
Jul 14, 2022
Merged

Merge CalBERT-rewrite with Main #1

merged 43 commits into from
Jul 14, 2022

Conversation

aditeyabaral
Copy link
Owner

CalBERT-rewrite

This set of changes includes a complete rewrite of CalBERT and includes classes for the model, dataset and the Siamese Pretrainer.

Changelog

  • Complete rewrite in the most popular deep learning framework (IMO), Pytorch allows CalBERT to be used more extensively and easily. Support added for CalBERT to be used as-is, or imported as a layer in your own models.
  • Separate Torch-based class for handling CalBERT dataset creation with negative sampling included as well
  • Siamese Pretrainer is now a new class by itself, allowing you to pre-train any Transformer using CalBERT's Siamese Pre-training. Supports checkpointing, different contrastive losses (some not in the paper!), saving and loading of models and more.
  • A simple train.py script to help beginners get started with training CalBERT
  • CalBERT can now be installed via pip

@aditeyabaral aditeyabaral merged commit 3c04cdf into main Jul 14, 2022
@aditeyabaral aditeyabaral deleted the rewrite branch July 14, 2022 14:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants