Merge r1.6.0 main (#3500)

* update branch

Signed-off-by: ericharper <[email protected]>

* minor updates for finetuning (#3455)

Signed-off-by: Jason <[email protected]>

* Fix hysterisis loading (#3460)

Signed-off-by: MaximumEntropy <[email protected]>

* Fix the tutorial notebooks bug (#3465)

* fix checkpoint loading and model config file

Signed-off-by: Yi Dong <[email protected]>

* fix style

Signed-off-by: Yi Dong <[email protected]>

* Test only if the model was trained on single GPU for accurate results. (#3470)

* Test only if the model was trained on single GPU for accurate results.

Signed-off-by: smajumdar <[email protected]>

* Test only if the model was trained on single GPU for accurate results.

Signed-off-by: smajumdar <[email protected]>

* Fix the errors/bugs in ASR with diarization tutorial (#3461)

* Initial commit

Signed-off-by: Taejin Park <[email protected]>

* fixed missing docstring

Signed-off-by: Taejin Park <[email protected]>

* Fixed main to r1.6.0

Signed-off-by: Taejin Park <[email protected]>

* Fixed pip install issues

Signed-off-by: Taejin Park <[email protected]>

* NMT documentation for bottleneck architecture (#3464)

* 1. Updated NMT doc to include bottleneck architecture.

Signed-off-by: Micha Livne <[email protected]>

* 1. Debugging.

Signed-off-by: Micha Livne <[email protected]>

* 1. Debugging.

Signed-off-by: Micha Livne <[email protected]>

* 1. Debugging.

Signed-off-by: Micha Livne <[email protected]>

* 1. Debugging.

Signed-off-by: Micha Livne <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* WFST Punct post fix + punct tutorial fixes (#3469)

* punct tutorial and wfst_post_process firx

Signed-off-by: ekmb <[email protected]>

* format

Signed-off-by: ekmb <[email protected]>

Co-authored-by: PeganovAnton <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>

* Process correctly label ids dataset parameter + standardize type of label ids model attribute + minor changes (error messages, typing) (#3471)

* Fix label ids dictionary type

Signed-off-by: PeganovAnton <[email protected]>

* Fix to_container method usage

Signed-off-by: PeganovAnton <[email protected]>

Co-authored-by: ekmb <[email protected]>

* file name fix - Segmentation tutorial (#3474)

* update file name

Signed-off-by: ekmb <[email protected]>

* update file name

Signed-off-by: ekmb <[email protected]>

* Patch fix for the multiple last checkpoints issue (#3468)

* fix line

Signed-off-by: nithinraok <[email protected]>

* add TODO comment

Signed-off-by: nithinraok <[email protected]>

* upper bound ptl, lower bound numpy (#3466)

Signed-off-by: ericharper <[email protected]>

* fix bug with arguments for preprocessor (#3481)

Signed-off-by: Oktai Tatanov <[email protected]>

* Fix `punctuation_capitalization_train_evaluate.py` description (#3482)

* fix run script documentation

Signed-off-by: PeganovAnton <[email protected]>

* Add missing parameters to examples in documentation

Signed-off-by: PeganovAnton <[email protected]>

* Standardize format of paths and file names in docs examples

Signed-off-by: PeganovAnton <[email protected]>

* Remove wordtokenizer example from NLP tokenizer notebook (#3477)

* nb fix

Signed-off-by: Abhinav Khattar <[email protected]>

* keep token change for later

Signed-off-by: Abhinav Khattar <[email protected]>

* fix

Signed-off-by: Abhinav Khattar <[email protected]>

* Properly support -1 for labels in ctc char models (#3487)

Signed-off-by: smajumdar <[email protected]>

Co-authored-by: Vahid Noroozi <[email protected]>

* typo fix in diarization notebooks (#3480)

Signed-off-by: nithinraok <[email protected]>

* Add Apex import guard (#3467)

* add apex guard

Signed-off-by: ericharper <[email protected]>

* add import guard

Signed-off-by: ericharper <[email protected]>

* remove MegatronBertEncoder class

Signed-off-by: ericharper <[email protected]>

* update warning message when apex not found

Signed-off-by: ericharper <[email protected]>

* style

Signed-off-by: ericharper <[email protected]>

* remove import from init

Signed-off-by: ericharper <[email protected]>

* remove unused import

Signed-off-by: ericharper <[email protected]>

* Fix checkpoint converter in O2 style (#3486)

* Fix checkpoint converter in O2 style

Signed-off-by: Yu Yao <[email protected]>

* Fix style

Signed-off-by: Yu Yao <[email protected]>

Co-authored-by: Yu Yao <[email protected]>
Co-authored-by: Eric Harper <[email protected]>

* update branch

Signed-off-by: ericharper <[email protected]>

Co-authored-by: Jason <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Taejin Park <[email protected]>
Co-authored-by: Micha Livne <[email protected]>
Co-authored-by: Micha Livne <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: PeganovAnton <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: ekmb <[email protected]>
Co-authored-by: Nithin Rao <[email protected]>
Co-authored-by: Oktai Tatanov <[email protected]>
Co-authored-by: Abhinav Khattar <[email protected]>
Co-authored-by: Vahid Noroozi <[email protected]>
Co-authored-by: yaoyu-33 <[email protected]>
Co-authored-by: Yu Yao <[email protected]>

requirements/requirements_lightning.txt

@@ -1,7 +1,7 @@
-pytorch-lightning==1.5.9
+pytorch-lightning>=1.5.0,<=1.5.8
 torchmetrics>=0.4.1rc0
 transformers>=4.0.1
 webdataset>=0.1.48,<=0.1.62
 omegaconf>=2.1.0
 hydra-core>=1.1.0
-pyyaml<6  # Pinned until omegaconf works with pyyaml>=6
+pyyaml<6  # Pinned until omegaconf works with pyyaml>=6