lm

Rowan Zellers

Jun 12, 2019

38f7184 · Jun 12, 2019

This branch is 4 commits behind rowanz/grover:master.

Name	Name	Last commit message	Last commit date
parent directory ..
configs	configs	Initial commit, now we can generate easily and stuff	Jun 12, 2019
README.md	README.md	Initial commit, now we can generate easily and stuff	Jun 12, 2019
__init__.py	__init__.py	Initial commit, now we can generate easily and stuff	Jun 12, 2019
dataloader.py	dataloader.py	Initial commit, now we can generate easily and stuff	Jun 12, 2019
modeling.py	modeling.py	Initial commit, now we can generate easily and stuff	Jun 12, 2019
optimization_adafactor.py	optimization_adafactor.py	Initial commit, now we can generate easily and stuff	Jun 12, 2019
train.py	train.py	Initial commit, now we can generate easily and stuff	Jun 12, 2019
train_tpu_adafactor.sh	train_tpu_adafactor.sh	Initial commit, now we can generate easily and stuff	Jun 12, 2019
utils.py	utils.py	Initial commit, now we can generate easily and stuff	Jun 12, 2019
validate.py	validate.py	Initial commit, now we can generate easily and stuff	Jun 12, 2019
validate.sh	validate.sh	Initial commit, now we can generate easily and stuff	Jun 12, 2019

README.md

validate.py gets perplexity. You can use the script validate.sh which contains some arguments I used.
train.py trains a grover model from scratch. The script train_tpu_adafactor.sh could also help. You probably don't want to do this unless you have a lot of money for TPUs. However, it might be a good idea to finetune Grover to a different domain.

During training, you can use tensorboard by running the following commands:

ssh -L 6006:localhost:6006 myservername
tensorboard --logdir="grover":"gs://MYOUTPUTPATH" --port=6006