Ignore me!

[davidefiocco]

I did this PR by mistake, and found out late. Sorry Hugging Face!

[davidefiocco]

Sorry for the mess-up, was working on a fork of mine (facepalm) and accidentally PR to the repo.

[julien-c]

No worries – but potentially open to a PR adding Azure logging if it makes sense on your end!

[davidefiocco]

Hi @julien-c , thanks for pointing that out!

I am running transformers on AzureML (finetuning on classification tasks mainly), as I find it convenient because of the e.g. automatic compute provisioning and downsizing, experiment tracking, model versioning capabilities Azure offers.

I don't know what's the best way to contribute with respect to this (i.e. sharing code running finetuning of BERT using AzureML), but here's what I know:

• Implementing tensorboard-like capabilities in AzureML experiments can be done with trivial modifications to the current trainer.py, along the lines of 939f679#diff-cd3b4dd7d09c0b32ff40ccc1840a4da4

• Those trivial changes allow to see in the AzureML frontend trends like

which can be handy when running experiments.

• To launch one experiment, one would use Azure-specific code like that in https://github.com/microsoft/AzureML-BERT/blob/master/finetune/PyTorch/notebooks/Pretrained-BERT-GLUE.ipynb

Microsoft had started a pretty cool repo at https://github.com/microsoft/AzureML-BERT/ with solutions for this using https://github.com/huggingface/transformers/tree/v0.6.2 . What I think is happening though is that transformers has evolved quite a lot since then and they would need to refresh those examples.

I honestly don't know if it'd make sense to PR https://github.com/microsoft/AzureML-BERT (and not clutter the transformers with platform-specific code). WDYT? Cheers!

[davidefiocco]

The idea above is resurrected here #8062