Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option for specifying wandb save_dir from config #4379

Merged
merged 3 commits into from
Jun 16, 2022

Conversation

shan18
Copy link
Member

@shan18 shan18 commented Jun 15, 2022

Signed-off-by: Shantanu Acharya [email protected]

What does this PR do ?

A user can now specify the name of the directory to save the wandb logs by setting the value exp_manager.wandb_logger_kwargs.save_dir in the config.

Collection: [utils]

Changelog

Files changed: nemo/utils/exp_manager.py (line: 670-671)

if wandb_kwargs.get('save_dir', None) is None:
    wandb_kwargs['save_dir'] = exp_dir
os.makedirs(wandb_kwargs['save_dir'], exist_ok=True)
wandb_logger = WandbLogger(version=version, **wandb_kwargs)

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?
  • Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
    • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

  • New Feature
  • Bugfix
  • Documentation

Who can review?

Anyone in NeMo

wandb_logger = WandbLogger(save_dir=exp_dir, version=version, **wandb_kwargs)

# Update the wandb save_dir
wandb_kwargs['save_dir'] = wandb_kwargs.get('save_dir') or exp_dir
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't do get(save_dir) do get(save_dir, None) is not None else exp_dir. Also don't automatically create dir, wandb should do it for you no ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, wandb is not creating a new directory if it does not exit. It simply creates a random dir within /tmp and starts writing logs there.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sad. Oh well. But still don't use get without a default v

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Updated.

@titu1994 titu1994 merged commit 17528bb into NVIDIA:main Jun 16, 2022
titu1994 pushed a commit to titu1994/NeMo that referenced this pull request Jun 21, 2022
* give option to user to specify wandb save dir via config

Signed-off-by: Shantanu Acharya <[email protected]>

* create save_dir directory for wandb logger if not exists

Signed-off-by: Shantanu Acharya <[email protected]>

* update save_dir get method with a default value

Signed-off-by: Shantanu Acharya <[email protected]>
jubick1337 pushed a commit to jubick1337/NeMo that referenced this pull request Jun 27, 2022
* give option to user to specify wandb save dir via config

Signed-off-by: Shantanu Acharya <[email protected]>

* create save_dir directory for wandb logger if not exists

Signed-off-by: Shantanu Acharya <[email protected]>

* update save_dir get method with a default value

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>
PeganovAnton pushed a commit that referenced this pull request Jul 7, 2022
* Dataloader, collector, loss and metric for multiscale diarization decoder  (#4187)

* First commit

Signed-off-by: Taejin Park <[email protected]>

* Checked funtionality and imports

Signed-off-by: Taejin Park <[email protected]>

* fixed import issues

Signed-off-by: Taejin Park <[email protected]>

* Removed the changed made by mistake

Signed-off-by: Taejin Park <[email protected]>

* Style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM errors 001

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM and style fix

Signed-off-by: Taejin Park <[email protected]>

* Changed docstrings

Signed-off-by: Taejin Park <[email protected]>

* LGTM again

Signed-off-by: Taejin Park <[email protected]>

* Removed unnecessary torch setting lines

Signed-off-by: Taejin Park <[email protected]>

* Style fix and isort

Signed-off-by: Taejin Park <[email protected]>

* jbalam-nv comments reflected

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected comments and created _diar_label.py

Signed-off-by: Taejin Park <[email protected]>

* Typo fix and style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed target_spks[0] index error

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* LGTM unused import IterDataset

Signed-off-by: Taejin Park <[email protected]>

* revert collection doc year

Signed-off-by: Taejin Park <[email protected]>

* Code format error in collections.py

Signed-off-by: Taejin Park <[email protected]>

* fix collections space format error

Signed-off-by: Taejin Park <[email protected]>

* merged main correctly

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected all comments and tested

Signed-off-by: Taejin Park <[email protected]>

* style fix and LGTM

Signed-off-by: Taejin Park <[email protected]>

* rttm_filepath to rttm_file and removed self included funcs, tested

Signed-off-by: Taejin Park <[email protected]>

Co-authored-by: Nithin Rao <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* removed references to data_dir

Signed-off-by: Matvei Novikov <[email protected]>

* added missing parameters to data preparation script

Signed-off-by: Matvei Novikov <[email protected]>

* removed unnecessary file extension check

Signed-off-by: Matvei Novikov <[email protected]>

* Add ASR CTC Decoding module (#4342)

* Initial commit

Signed-off-by: smajumdar <[email protected]>

* Full support for decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Temp

Signed-off-by: smajumdar <[email protected]>

* Fix labels of y_sequence

Signed-off-by: smajumdar <[email protected]>

* Set support for sentencepiece subword merging

Signed-off-by: smajumdar <[email protected]>

* Fix char and word based token merge alignment

Signed-off-by: smajumdar <[email protected]>

* Revert incorrect change

Signed-off-by: smajumdar <[email protected]>

* Update docstring

Signed-off-by: smajumdar <[email protected]>

* Improve compatibility with greedy tokens and log probs

Signed-off-by: smajumdar <[email protected]>

* Update scripts to use decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix decoding of ctc models

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Option to disable mp in VAD via num_workers=1 (#4317)

* Option to disable mp in VAD via num_workers=1

In certain environments python multiprocessing can deadlock. This adds a convenient version to disable by setting num_workers to 1.

Signed-off-by: Georg Kucsko <[email protected]>

* add none handling

Signed-off-by: Georg Kucsko <[email protected]>

* additional none handling

Signed-off-by: Georg Kucsko <[email protected]>

Co-authored-by: fayejf <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* remove redundant bias expand (#4382)

* remove redundant bias expand

Signed-off-by: Xiaowei Ren <[email protected]>

* delete redundant code

Signed-off-by: Xiaowei Ren <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* fixed style

Signed-off-by: Matvei Novikov <[email protected]>

* Add option for specifying wandb save_dir from config (#4379)

* give option to user to specify wandb save dir via config

Signed-off-by: Shantanu Acharya <[email protected]>

* create save_dir directory for wandb logger if not exists

Signed-off-by: Shantanu Acharya <[email protected]>

* update save_dir get method with a default value

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* [Bugfix][TTS] wrong order of returned tuple for general_collate_fn. (#4388)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Merge r1.10.0 main (#4398)

* update branch

Signed-off-by: ericharper <[email protected]>

* Set headscale false (#4364)

Signed-off-by: MaximumEntropy <[email protected]>

* Add wandb as dependency (#4365)

Signed-off-by: smajumdar <[email protected]>

* Raise trainer error (#4356)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Set headscale false (#4364) (#4366)

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: smajumdar <[email protected]>

* Finetuning changes for BART (#4003)

* Temp

Signed-off-by: MaximumEntropy <[email protected]>

* Checkpoint converter to nemo for bart

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Make position embedding expansion specific to a batch to avoid checkpoint size mismatches (#4357)

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Fix logging warning

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Fix electronic bug, new time ITN rule (#4355)

* fix electronic bug

Signed-off-by: ekmb <[email protected]>

* add new itn time rule

Signed-off-by: ekmb <[email protected]>

* revert domain changes

Signed-off-by: ekmb <[email protected]>

* remove repetition

Signed-off-by: ekmb <[email protected]>

* Correct support for dataclasses in default module dim (#4372)

* Correct support for dataclasses in default module dim

Signed-off-by: smajumdar <[email protected]>

* Fix path for save of results

Signed-off-by: smajumdar <[email protected]>

* fix pad id bug (#4377)

Signed-off-by: Yi Dong <[email protected]>

* Question answering bug fix (#4381)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assistant data processsor

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins for SGDGEN local checkpoint

Signed-off-by: Zhilin Wang <[email protected]>

* update style

Signed-off-by: Zhilin Wang <[email protected]>

* use local vocab file for Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* patch for Jenkins CI using local file

Signed-off-by: Zhilin Wang <[email protected]>

* add slot filling prediction and metrics

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused code

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* refactor metrics code out of Dialogue GPT Model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate backward compatible support for IntentSlotClassificationModel (bert model)

Signed-off-by: Zhilin Wang <[email protected]>

* save prediction file for IntentSlotClassification

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue gpt model training for megatron gpt

Signed-off-by: Zhilin Wang <[email protected]>

* remove batch generate for HF GPT2, which causes lower performance

Signed-off-by: Zhilin Wang <[email protected]>

* add few shot capability to dialogue gpt model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile and remove unused import

Signed-off-by: Zhilin Wang <[email protected]>

* update code description and clarity

Signed-off-by: Zhilin Wang <[email protected]>

* address PR comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate compatibility with ZeroShotIntentModel

Signed-off-by: Zhilin Wang <[email protected]>

* rename folder to dialogue due to increased scope and further refactor for clarity

Signed-off-by: Zhilin Wang <[email protected]>

* added dialogue GPT for sequence generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* add CI test for DialogueGPTGenerationModel

Signed-off-by: Zhilin Wang <[email protected]>

* integrate DialogueS2SGenerationModel for generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* modify huggingface utils to support HF t5/BART models

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* fix bleu metric style

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* update based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 2 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 3 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* integrate sgd generation based on user user utterance and system slot-values to generate system utterance

Signed-off-by: Zhilin Wang <[email protected]>

* add validation model saving capabilities

Signed-off-by: Zhilin Wang <[email protected]>

* cleaned up code for SGD Based Answer extender

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue Generation CI

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* fix Jenkins CI issue"

Signed-off-by: Zhilin Wang <[email protected]>

* add support for design dataset

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary imports

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support megatron for dialogue_s2s_generation_model

Signed-off-by: Zhilin Wang <[email protected]>

* reduce loaded samples in MSMarcoDataProcessor to 64 when cfg.model.dataset.debug_mode=True

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update CI

Signed-off-by: Zhilin Wang <[email protected]>

* update checkpoint and predictions filename to include epoch number

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate HF BART MNLI into zero shot intent model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Nearest Neighbour Model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* refactor Dialogue SGD Data Processor to make interface for models cleaner

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue S2S Generation model for DialogueSGDDataProcessor interface

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support sgd and drive thru datasets by zero shot model and nearest neighbour model

Signed-off-by: Zhilin Wang <[email protected]>

* add prediction saving code to nearest neighbour and zero shot intent models

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo in sgd data processor

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Mellon QA Data Processor

Signed-off-by: Zhilin Wang <[email protected]>

* update mellon qa

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue.py to remove outdated info

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* add dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* address review comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix for cfg

Signed-off-by: Zhilin Wang <[email protected]>

* make dependency on apex optional

Signed-off-by: Zhilin Wang <[email protected]>

* change NLPDDPluggin calling logic to make it possible to run without apex

Signed-off-by: Zhilin Wang <[email protected]>

* add first draft of tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* reduce ms marco size by removing lines without wellFormedAnswers

Signed-off-by: Zhilin Wang <[email protected]>

* address pr comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update colab tutorial link in dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* include unit test and some refactor to facilitate unit test

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* address pr issues

Signed-off-by: Zhilin Wang <[email protected]>

* remove typos in dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* support larger files for question answering

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary artifacts to reduce memory use

Signed-off-by: Zhilin Wang <[email protected]>

* put 0 tensor to device

Signed-off-by: Zhilin Wang <[email protected]>

* update link within dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously delete files

Signed-off-by: Zhilin Wang <[email protected]>

* update error handling when loss = nan

Signed-off-by: Zhilin Wang <[email protected]>

* update nan handling

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss func

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss

Signed-off-by: Zhilin Wang <[email protected]>

* fix type error raised in qa_dataset.py

Signed-off-by: Zhilin Wang <[email protected]>

* add error checking message

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update exp logging

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* limit number of negative samples

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused methods and style fix

Signed-off-by: Zhilin Wang <[email protected]>

* add more documentation

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* changes base on PR review

Signed-off-by: Zhilin Wang <[email protected]>

* set wandb logger falseby default

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

* style fix

* correct typo

* style fix

* style fix

Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>

* Fix ASR Typos in tutorials (#4384)

* Fix typos

Signed-off-by: smajumdar <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
(cherry picked from commit 0322b15)

Co-authored-by: Travis Bartley <[email protected]>

* Add Docs for NeMo Adapters (#4369)

Signed-off-by: smajumdar <[email protected]>

* Update NeMo docs (#4397)

Signed-off-by: smajumdar <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* update branch

Signed-off-by: ericharper <[email protected]>

* remove Copy of

Signed-off-by: ericharper <[email protected]>

Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Micha Livne <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Travis Bartley <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* [bugfix][TTS] pitch, voiced_mask, prob_voiced have the same values. (#4392)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Fixing import error in some cases (#4401)

Signed-off-by: Boris Fomitchev <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Fixing bugs in calling method ctc_decoder_predictions_tensor. (#4414)

* updated ctc decoding calls.

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Update with new conformer checkpoints. (#4417)

Signed-off-by: Matvei Novikov <[email protected]>

* [TTS] add static method decorator. (#4443)

* [TTS] add static method decorator.

Signed-off-by: Xuesong Yang <[email protected]>

* remove protect prefix

Signed-off-by: Xuesong Yang <[email protected]>

* fixed style error

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

Co-authored-by: Taejin Park <[email protected]>
Co-authored-by: Nithin Rao <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Georg Kucsko <[email protected]>
Co-authored-by: fayejf <[email protected]>
Co-authored-by: Xiaowei Ren <[email protected]>
Co-authored-by: Shantanu Acharya <[email protected]>
Co-authored-by: Travis Bartley <[email protected]>
Co-authored-by: tbartley94 <[email protected]>
Co-authored-by: Xuesong Yang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Micha Livne <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Boris Fomitchev <[email protected]>
Co-authored-by: Vahid Noroozi <[email protected]>
arendu pushed a commit that referenced this pull request Jul 21, 2022
* Dataloader, collector, loss and metric for multiscale diarization decoder  (#4187)

* First commit

Signed-off-by: Taejin Park <[email protected]>

* Checked funtionality and imports

Signed-off-by: Taejin Park <[email protected]>

* fixed import issues

Signed-off-by: Taejin Park <[email protected]>

* Removed the changed made by mistake

Signed-off-by: Taejin Park <[email protected]>

* Style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM errors 001

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM and style fix

Signed-off-by: Taejin Park <[email protected]>

* Changed docstrings

Signed-off-by: Taejin Park <[email protected]>

* LGTM again

Signed-off-by: Taejin Park <[email protected]>

* Removed unnecessary torch setting lines

Signed-off-by: Taejin Park <[email protected]>

* Style fix and isort

Signed-off-by: Taejin Park <[email protected]>

* jbalam-nv comments reflected

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected comments and created _diar_label.py

Signed-off-by: Taejin Park <[email protected]>

* Typo fix and style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed target_spks[0] index error

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* LGTM unused import IterDataset

Signed-off-by: Taejin Park <[email protected]>

* revert collection doc year

Signed-off-by: Taejin Park <[email protected]>

* Code format error in collections.py

Signed-off-by: Taejin Park <[email protected]>

* fix collections space format error

Signed-off-by: Taejin Park <[email protected]>

* merged main correctly

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected all comments and tested

Signed-off-by: Taejin Park <[email protected]>

* style fix and LGTM

Signed-off-by: Taejin Park <[email protected]>

* rttm_filepath to rttm_file and removed self included funcs, tested

Signed-off-by: Taejin Park <[email protected]>

Co-authored-by: Nithin Rao <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* removed references to data_dir

Signed-off-by: Matvei Novikov <[email protected]>

* added missing parameters to data preparation script

Signed-off-by: Matvei Novikov <[email protected]>

* removed unnecessary file extension check

Signed-off-by: Matvei Novikov <[email protected]>

* Add ASR CTC Decoding module (#4342)

* Initial commit

Signed-off-by: smajumdar <[email protected]>

* Full support for decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Temp

Signed-off-by: smajumdar <[email protected]>

* Fix labels of y_sequence

Signed-off-by: smajumdar <[email protected]>

* Set support for sentencepiece subword merging

Signed-off-by: smajumdar <[email protected]>

* Fix char and word based token merge alignment

Signed-off-by: smajumdar <[email protected]>

* Revert incorrect change

Signed-off-by: smajumdar <[email protected]>

* Update docstring

Signed-off-by: smajumdar <[email protected]>

* Improve compatibility with greedy tokens and log probs

Signed-off-by: smajumdar <[email protected]>

* Update scripts to use decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix decoding of ctc models

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Option to disable mp in VAD via num_workers=1 (#4317)

* Option to disable mp in VAD via num_workers=1

In certain environments python multiprocessing can deadlock. This adds a convenient version to disable by setting num_workers to 1.

Signed-off-by: Georg Kucsko <[email protected]>

* add none handling

Signed-off-by: Georg Kucsko <[email protected]>

* additional none handling

Signed-off-by: Georg Kucsko <[email protected]>

Co-authored-by: fayejf <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* remove redundant bias expand (#4382)

* remove redundant bias expand

Signed-off-by: Xiaowei Ren <[email protected]>

* delete redundant code

Signed-off-by: Xiaowei Ren <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* fixed style

Signed-off-by: Matvei Novikov <[email protected]>

* Add option for specifying wandb save_dir from config (#4379)

* give option to user to specify wandb save dir via config

Signed-off-by: Shantanu Acharya <[email protected]>

* create save_dir directory for wandb logger if not exists

Signed-off-by: Shantanu Acharya <[email protected]>

* update save_dir get method with a default value

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* [Bugfix][TTS] wrong order of returned tuple for general_collate_fn. (#4388)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Merge r1.10.0 main (#4398)

* update branch

Signed-off-by: ericharper <[email protected]>

* Set headscale false (#4364)

Signed-off-by: MaximumEntropy <[email protected]>

* Add wandb as dependency (#4365)

Signed-off-by: smajumdar <[email protected]>

* Raise trainer error (#4356)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Set headscale false (#4364) (#4366)

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: smajumdar <[email protected]>

* Finetuning changes for BART (#4003)

* Temp

Signed-off-by: MaximumEntropy <[email protected]>

* Checkpoint converter to nemo for bart

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Make position embedding expansion specific to a batch to avoid checkpoint size mismatches (#4357)

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Fix logging warning

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Fix electronic bug, new time ITN rule (#4355)

* fix electronic bug

Signed-off-by: ekmb <[email protected]>

* add new itn time rule

Signed-off-by: ekmb <[email protected]>

* revert domain changes

Signed-off-by: ekmb <[email protected]>

* remove repetition

Signed-off-by: ekmb <[email protected]>

* Correct support for dataclasses in default module dim (#4372)

* Correct support for dataclasses in default module dim

Signed-off-by: smajumdar <[email protected]>

* Fix path for save of results

Signed-off-by: smajumdar <[email protected]>

* fix pad id bug (#4377)

Signed-off-by: Yi Dong <[email protected]>

* Question answering bug fix (#4381)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assistant data processsor

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins for SGDGEN local checkpoint

Signed-off-by: Zhilin Wang <[email protected]>

* update style

Signed-off-by: Zhilin Wang <[email protected]>

* use local vocab file for Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* patch for Jenkins CI using local file

Signed-off-by: Zhilin Wang <[email protected]>

* add slot filling prediction and metrics

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused code

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* refactor metrics code out of Dialogue GPT Model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate backward compatible support for IntentSlotClassificationModel (bert model)

Signed-off-by: Zhilin Wang <[email protected]>

* save prediction file for IntentSlotClassification

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue gpt model training for megatron gpt

Signed-off-by: Zhilin Wang <[email protected]>

* remove batch generate for HF GPT2, which causes lower performance

Signed-off-by: Zhilin Wang <[email protected]>

* add few shot capability to dialogue gpt model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile and remove unused import

Signed-off-by: Zhilin Wang <[email protected]>

* update code description and clarity

Signed-off-by: Zhilin Wang <[email protected]>

* address PR comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate compatibility with ZeroShotIntentModel

Signed-off-by: Zhilin Wang <[email protected]>

* rename folder to dialogue due to increased scope and further refactor for clarity

Signed-off-by: Zhilin Wang <[email protected]>

* added dialogue GPT for sequence generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* add CI test for DialogueGPTGenerationModel

Signed-off-by: Zhilin Wang <[email protected]>

* integrate DialogueS2SGenerationModel for generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* modify huggingface utils to support HF t5/BART models

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* fix bleu metric style

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* update based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 2 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 3 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* integrate sgd generation based on user user utterance and system slot-values to generate system utterance

Signed-off-by: Zhilin Wang <[email protected]>

* add validation model saving capabilities

Signed-off-by: Zhilin Wang <[email protected]>

* cleaned up code for SGD Based Answer extender

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue Generation CI

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* fix Jenkins CI issue"

Signed-off-by: Zhilin Wang <[email protected]>

* add support for design dataset

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary imports

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support megatron for dialogue_s2s_generation_model

Signed-off-by: Zhilin Wang <[email protected]>

* reduce loaded samples in MSMarcoDataProcessor to 64 when cfg.model.dataset.debug_mode=True

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update CI

Signed-off-by: Zhilin Wang <[email protected]>

* update checkpoint and predictions filename to include epoch number

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate HF BART MNLI into zero shot intent model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Nearest Neighbour Model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* refactor Dialogue SGD Data Processor to make interface for models cleaner

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue S2S Generation model for DialogueSGDDataProcessor interface

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support sgd and drive thru datasets by zero shot model and nearest neighbour model

Signed-off-by: Zhilin Wang <[email protected]>

* add prediction saving code to nearest neighbour and zero shot intent models

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo in sgd data processor

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Mellon QA Data Processor

Signed-off-by: Zhilin Wang <[email protected]>

* update mellon qa

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue.py to remove outdated info

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* add dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* address review comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix for cfg

Signed-off-by: Zhilin Wang <[email protected]>

* make dependency on apex optional

Signed-off-by: Zhilin Wang <[email protected]>

* change NLPDDPluggin calling logic to make it possible to run without apex

Signed-off-by: Zhilin Wang <[email protected]>

* add first draft of tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* reduce ms marco size by removing lines without wellFormedAnswers

Signed-off-by: Zhilin Wang <[email protected]>

* address pr comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update colab tutorial link in dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* include unit test and some refactor to facilitate unit test

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* address pr issues

Signed-off-by: Zhilin Wang <[email protected]>

* remove typos in dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* support larger files for question answering

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary artifacts to reduce memory use

Signed-off-by: Zhilin Wang <[email protected]>

* put 0 tensor to device

Signed-off-by: Zhilin Wang <[email protected]>

* update link within dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously delete files

Signed-off-by: Zhilin Wang <[email protected]>

* update error handling when loss = nan

Signed-off-by: Zhilin Wang <[email protected]>

* update nan handling

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss func

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss

Signed-off-by: Zhilin Wang <[email protected]>

* fix type error raised in qa_dataset.py

Signed-off-by: Zhilin Wang <[email protected]>

* add error checking message

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update exp logging

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* limit number of negative samples

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused methods and style fix

Signed-off-by: Zhilin Wang <[email protected]>

* add more documentation

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* changes base on PR review

Signed-off-by: Zhilin Wang <[email protected]>

* set wandb logger falseby default

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

* style fix

* correct typo

* style fix

* style fix

Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>

* Fix ASR Typos in tutorials (#4384)

* Fix typos

Signed-off-by: smajumdar <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
(cherry picked from commit 0322b15)

Co-authored-by: Travis Bartley <[email protected]>

* Add Docs for NeMo Adapters (#4369)

Signed-off-by: smajumdar <[email protected]>

* Update NeMo docs (#4397)

Signed-off-by: smajumdar <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* update branch

Signed-off-by: ericharper <[email protected]>

* remove Copy of

Signed-off-by: ericharper <[email protected]>

Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Micha Livne <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Travis Bartley <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* [bugfix][TTS] pitch, voiced_mask, prob_voiced have the same values. (#4392)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Fixing import error in some cases (#4401)

Signed-off-by: Boris Fomitchev <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Fixing bugs in calling method ctc_decoder_predictions_tensor. (#4414)

* updated ctc decoding calls.

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Update with new conformer checkpoints. (#4417)

Signed-off-by: Matvei Novikov <[email protected]>

* [TTS] add static method decorator. (#4443)

* [TTS] add static method decorator.

Signed-off-by: Xuesong Yang <[email protected]>

* remove protect prefix

Signed-off-by: Xuesong Yang <[email protected]>

* fixed style error

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

Co-authored-by: Taejin Park <[email protected]>
Co-authored-by: Nithin Rao <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Georg Kucsko <[email protected]>
Co-authored-by: fayejf <[email protected]>
Co-authored-by: Xiaowei Ren <[email protected]>
Co-authored-by: Shantanu Acharya <[email protected]>
Co-authored-by: Travis Bartley <[email protected]>
Co-authored-by: tbartley94 <[email protected]>
Co-authored-by: Xuesong Yang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Micha Livne <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Boris Fomitchev <[email protected]>
Co-authored-by: Vahid Noroozi <[email protected]>
Signed-off-by: arendu <[email protected]>
paarthneekhara added a commit to paarthneekhara/NeMo that referenced this pull request Jul 24, 2022
* Megatron BART BOS / EOS bug fix (#4495)

* 1. Debugging.

Signed-off-by: Micha Livne <[email protected]>

* 1. BART dataset fixes missing <EOS> for deocder output.

Signed-off-by: Micha Livne <[email protected]>

* 1. Debugging.

Signed-off-by: Micha Livne <[email protected]>

* 1. Debugging.

Signed-off-by: Micha Livne <[email protected]>

* 1. Removed extra padding from BARTDataset.

Signed-off-by: Micha Livne <[email protected]>

* GPT Prompt Learning Improvements (#4496)

* Updated pipeline parallel code to speed up training

Signed-off-by: Virginia Adams <[email protected]>

* Load global batch size not local mini batch size

Signed-off-by: Virginia Adams <[email protected]>

* Python reformatting

Signed-off-by: Virginia Adams <[email protected]>

* Megatron perceiver with tensor parallelism only (#4318)

* Temp

Signed-off-by: MaximumEntropy <[email protected]>

* Add megatron dataset

Signed-off-by: MaximumEntropy <[email protected]>

* Update config and fix global batch fetcher

Signed-off-by: MaximumEntropy <[email protected]>

* Add dataset class

Signed-off-by: MaximumEntropy <[email protected]>

* Update comments

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Update yaml

Signed-off-by: MaximumEntropy <[email protected]>

* Fix duplicate yaml key

Signed-off-by: MaximumEntropy <[email protected]>

* Translate method and preprocess script for raw text

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Remove pdb

Signed-off-by: MaximumEntropy <[email protected]>

* Fix arg name

Signed-off-by: MaximumEntropy <[email protected]>

* Fix other arg

Signed-off-by: MaximumEntropy <[email protected]>

* Change sampler back

Signed-off-by: MaximumEntropy <[email protected]>

* Move back to global batch fetcher to use distributed sampler

Signed-off-by: MaximumEntropy <[email protected]>

* Add text memmap data

Signed-off-by: MaximumEntropy <[email protected]>

* Update monitor

Signed-off-by: MaximumEntropy <[email protected]>

* Fixes for PP

Signed-off-by: MaximumEntropy <[email protected]>

* Remove unused import

Signed-off-by: MaximumEntropy <[email protected]>

* Truncate examples in text memmap

Signed-off-by: MaximumEntropy <[email protected]>

* NMT training batch interpolation key

Signed-off-by: MaximumEntropy <[email protected]>

* tarred data fix

Signed-off-by: MaximumEntropy <[email protected]>

* Change dataset type check

Signed-off-by: MaximumEntropy <[email protected]>

* Fix sampler

Signed-off-by: MaximumEntropy <[email protected]>

* Pass dataset cfg to determine type

Signed-off-by: MaximumEntropy <[email protected]>

* Log global step on validation step as well

Signed-off-by: MaximumEntropy <[email protected]>

* Fix NMT model saving with artifacts

Signed-off-by: MaximumEntropy <[email protected]>

* Initialize DDP in decode if not initialized. Needed for inference only mode

Signed-off-by: MaximumEntropy <[email protected]>

* Megatron NMT inference script

Signed-off-by: MaximumEntropy <[email protected]>

* Inference config file

Signed-off-by: MaximumEntropy <[email protected]>

* hardcode max delta temporarily

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* detokenizer if processor is not none

Signed-off-by: MaximumEntropy <[email protected]>

* Sampler config

Signed-off-by: MaximumEntropy <[email protected]>

* Compat with configs without sampler arg

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Comment for validation dataset type

Signed-off-by: MaximumEntropy <[email protected]>

* Fix tokenizer building

Signed-off-by: MaximumEntropy <[email protected]>

* CI test for megatron nmt

Signed-off-by: MaximumEntropy <[email protected]>

* Fix tokenizer in restore

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* O2 restore from fix

Signed-off-by: MaximumEntropy <[email protected]>

* Remove print

Signed-off-by: MaximumEntropy <[email protected]>

* Change tokenizer model name in config

Signed-off-by: MaximumEntropy <[email protected]>

* Logging

Signed-off-by: MaximumEntropy <[email protected]>

* Set seed for distributed sampler

Signed-off-by: MaximumEntropy <[email protected]>

* Cluster debugging messages

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Fix max generation delta

Signed-off-by: MaximumEntropy <[email protected]>

* No LM Init

Signed-off-by: MaximumEntropy <[email protected]>

* Use nlp save restore connector

Signed-off-by: MaximumEntropy <[email protected]>

* Remove useless infer args

Signed-off-by: MaximumEntropy <[email protected]>

* Typo

Signed-off-by: MaximumEntropy <[email protected]>

* UTF8 safe print of translation result

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Add save restore connector back with comment

Signed-off-by: MaximumEntropy <[email protected]>

* Refactor

Signed-off-by: MaximumEntropy <[email protected]>

* Fix CI test

Signed-off-by: MaximumEntropy <[email protected]>

* Add missing args

Signed-off-by: MaximumEntropy <[email protected]>

* Address comments

Signed-off-by: MaximumEntropy <[email protected]>

* Empty to restart

* Fix CI test

Signed-off-by: MaximumEntropy <[email protected]>

* Check for test ds

Signed-off-by: MaximumEntropy <[email protected]>

* set fusion to false

Signed-off-by: MaximumEntropy <[email protected]>

* Initial perceiver encoder

Signed-off-by: MaximumEntropy <[email protected]>

* Perceiver with PP=1

Signed-off-by: MaximumEntropy <[email protected]>

* Remove init cross attn

Signed-off-by: MaximumEntropy <[email protected]>

* CI test and remove init cross attn arg

Signed-off-by: MaximumEntropy <[email protected]>

* Remove init cross attn layers from file

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Clean up

Signed-off-by: MaximumEntropy <[email protected]>

* update branch

Signed-off-by: ericharper <[email protected]>

* Set headscale false (#4364)

Signed-off-by: MaximumEntropy <[email protected]>

* Add wandb as dependency (#4365)

Signed-off-by: smajumdar <[email protected]>

* Raise trainer error (#4356)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Set headscale false (#4364) (#4366)

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: smajumdar <[email protected]>

* Finetuning changes for BART (#4003)

* Temp

Signed-off-by: MaximumEntropy <[email protected]>

* Checkpoint converter to nemo for bart

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Make position embedding expansion specific to a batch to avoid checkpoint size mismatches (#4357)

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Fix logging warning

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Refactor bias act fusion

Signed-off-by: MaximumEntropy <[email protected]>

* Update NMT config

Signed-off-by: MaximumEntropy <[email protected]>

* Fix electronic bug, new time ITN rule (#4355)

* fix electronic bug

Signed-off-by: ekmb <[email protected]>

* add new itn time rule

Signed-off-by: ekmb <[email protected]>

* revert domain changes

Signed-off-by: ekmb <[email protected]>

* remove repetition

Signed-off-by: ekmb <[email protected]>

* Update ci tests

Signed-off-by: MaximumEntropy <[email protected]>

* Correct support for dataclasses in default module dim (#4372)

* Correct support for dataclasses in default module dim

Signed-off-by: smajumdar <[email protected]>

* Fix path for save of results

Signed-off-by: smajumdar <[email protected]>

* fix pad id bug (#4377)

Signed-off-by: Yi Dong <[email protected]>

* Question answering bug fix (#4381)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assistant data processsor

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins for SGDGEN local checkpoint

Signed-off-by: Zhilin Wang <[email protected]>

* update style

Signed-off-by: Zhilin Wang <[email protected]>

* use local vocab file for Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* patch for Jenkins CI using local file

Signed-off-by: Zhilin Wang <[email protected]>

* add slot filling prediction and metrics

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused code

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* refactor metrics code out of Dialogue GPT Model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate backward compatible support for IntentSlotClassificationModel (bert model)

Signed-off-by: Zhilin Wang <[email protected]>

* save prediction file for IntentSlotClassification

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue gpt model training for megatron gpt

Signed-off-by: Zhilin Wang <[email protected]>

* remove batch generate for HF GPT2, which causes lower performance

Signed-off-by: Zhilin Wang <[email protected]>

* add few shot capability to dialogue gpt model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile and remove unused import

Signed-off-by: Zhilin Wang <[email protected]>

* update code description and clarity

Signed-off-by: Zhilin Wang <[email protected]>

* address PR comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate compatibility with ZeroShotIntentModel

Signed-off-by: Zhilin Wang <[email protected]>

* rename folder to dialogue due to increased scope and further refactor for clarity

Signed-off-by: Zhilin Wang <[email protected]>

* added dialogue GPT for sequence generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* add CI test for DialogueGPTGenerationModel

Signed-off-by: Zhilin Wang <[email protected]>

* integrate DialogueS2SGenerationModel for generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* modify huggingface utils to support HF t5/BART models

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* fix bleu metric style

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* update based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 2 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 3 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* integrate sgd generation based on user user utterance and system slot-values to generate system utterance

Signed-off-by: Zhilin Wang <[email protected]>

* add validation model saving capabilities

Signed-off-by: Zhilin Wang <[email protected]>

* cleaned up code for SGD Based Answer extender

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue Generation CI

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* fix Jenkins CI issue"

Signed-off-by: Zhilin Wang <[email protected]>

* add support for design dataset

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary imports

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support megatron for dialogue_s2s_generation_model

Signed-off-by: Zhilin Wang <[email protected]>

* reduce loaded samples in MSMarcoDataProcessor to 64 when cfg.model.dataset.debug_mode=True

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update CI

Signed-off-by: Zhilin Wang <[email protected]>

* update checkpoint and predictions filename to include epoch number

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate HF BART MNLI into zero shot intent model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Nearest Neighbour Model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* refactor Dialogue SGD Data Processor to make interface for models cleaner

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue S2S Generation model for DialogueSGDDataProcessor interface

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support sgd and drive thru datasets by zero shot model and nearest neighbour model

Signed-off-by: Zhilin Wang <[email protected]>

* add prediction saving code to nearest neighbour and zero shot intent models

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo in sgd data processor

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Mellon QA Data Processor

Signed-off-by: Zhilin Wang <[email protected]>

* update mellon qa

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue.py to remove outdated info

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* add dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* address review comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix for cfg

Signed-off-by: Zhilin Wang <[email protected]>

* make dependency on apex optional

Signed-off-by: Zhilin Wang <[email protected]>

* change NLPDDPluggin calling logic to make it possible to run without apex

Signed-off-by: Zhilin Wang <[email protected]>

* add first draft of tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* reduce ms marco size by removing lines without wellFormedAnswers

Signed-off-by: Zhilin Wang <[email protected]>

* address pr comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update colab tutorial link in dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* include unit test and some refactor to facilitate unit test

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* address pr issues

Signed-off-by: Zhilin Wang <[email protected]>

* remove typos in dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* support larger files for question answering

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary artifacts to reduce memory use

Signed-off-by: Zhilin Wang <[email protected]>

* put 0 tensor to device

Signed-off-by: Zhilin Wang <[email protected]>

* update link within dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously delete files

Signed-off-by: Zhilin Wang <[email protected]>

* update error handling when loss = nan

Signed-off-by: Zhilin Wang <[email protected]>

* update nan handling

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss func

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss

Signed-off-by: Zhilin Wang <[email protected]>

* fix type error raised in qa_dataset.py

Signed-off-by: Zhilin Wang <[email protected]>

* add error checking message

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update exp logging

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* limit number of negative samples

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused methods and style fix

Signed-off-by: Zhilin Wang <[email protected]>

* add more documentation

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* changes base on PR review

Signed-off-by: Zhilin Wang <[email protected]>

* set wandb logger falseby default

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

* style fix

* correct typo

* style fix

* style fix

Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>

* Fix ASR Typos in tutorials (#4384)

* Fix typos

Signed-off-by: smajumdar <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
(cherry picked from commit 0322b158f26a0b690edca7a84714e33752283923)

Co-authored-by: Travis Bartley <[email protected]>

* Add Docs for NeMo Adapters (#4369)

Signed-off-by: smajumdar <[email protected]>

* Update NeMo docs (#4397)

Signed-off-by: smajumdar <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* Punctuation and capitalization tests race condition (#4399)

* Add draft of race condition fixes

Signed-off-by: PeganovAnton <[email protected]>

* Minor improvements

Signed-off-by: PeganovAnton <[email protected]>

* More race condition fixes

Signed-off-by: PeganovAnton <[email protected]>

* Improve error message

Signed-off-by: PeganovAnton <[email protected]>

* Improve error message

Signed-off-by: PeganovAnton <[email protected]>

* Improve error message

Signed-off-by: PeganovAnton <[email protected]>

* bias act fusion changes

Signed-off-by: MaximumEntropy <[email protected]>

* Address comments

Signed-off-by: MaximumEntropy <[email protected]>

* Fix geglu without fusion

Signed-off-by: MaximumEntropy <[email protected]>

* Reset files to main

Signed-off-by: MaximumEntropy <[email protected]>

* Remove hidden blocks

Signed-off-by: MaximumEntropy <[email protected]>

* Fix style

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>
Co-authored-by: Abhinav Khattar <[email protected]>
Co-authored-by: ericharper <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Travis Bartley <[email protected]>
Co-authored-by: PeganovAnton <[email protected]>

* NMESC speaker counting algorithm update (#4500)

* initial commit

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Default maj_vote = False, max_rp=0.25

Signed-off-by: Taejin Park <[email protected]>

* doc strings and style fix

Signed-off-by: Taejin Park <[email protected]>

* Docstring minor edit

Signed-off-by: Taejin Park <[email protected]>

* Default False in the functions

Signed-off-by: Taejin Park <[email protected]>

* fixed repeated variable

Signed-off-by: Taejin Park <[email protected]>

* Default as maj_vote=False

Signed-off-by: Taejin Park <[email protected]>

* removed redundant part in wrtie_rttm func

Signed-off-by: Taejin Park <[email protected]>

* Removed unused function

Signed-off-by: Taejin Park <[email protected]>

* Updated and tested silence and very short samples

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Style fix and removing unnecessary parts

Signed-off-by: Taejin Park <[email protected]>

* unused variables are removed

Signed-off-by: Taejin Park <[email protected]>

* Fixed commented torch.jit.script

Signed-off-by: Taejin Park <[email protected]>

* majority voting update

Signed-off-by: Taejin Park <[email protected]>

* cancelling the update on speaker_utils and clus_diarizer

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* bug fix

Signed-off-by: Taejin Park <[email protected]>

* Added fp32 converting for torch.mm

Signed-off-by: Taejin Park <[email protected]>

Co-authored-by: Nithin Rao <[email protected]>

* Fix dataset parameter typo on tacotron2 example yaml (#4471)

Signed-off-by: saarus72 <[email protected]>

Co-authored-by: Xuesong Yang <[email protected]>

* Noam lr sched: do not force min_lr after max_steps (#4472)

Signed-off-by: Adrian Lancucki <[email protected]>

Co-authored-by: Adrian Lancucki <[email protected]>
Co-authored-by: Xuesong Yang <[email protected]>

* Refactor for punctuation model (#4367)

* Dataloader, collector, loss and metric for multiscale diarization decoder  (#4187)

* First commit

Signed-off-by: Taejin Park <[email protected]>

* Checked funtionality and imports

Signed-off-by: Taejin Park <[email protected]>

* fixed import issues

Signed-off-by: Taejin Park <[email protected]>

* Removed the changed made by mistake

Signed-off-by: Taejin Park <[email protected]>

* Style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM errors 001

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM and style fix

Signed-off-by: Taejin Park <[email protected]>

* Changed docstrings

Signed-off-by: Taejin Park <[email protected]>

* LGTM again

Signed-off-by: Taejin Park <[email protected]>

* Removed unnecessary torch setting lines

Signed-off-by: Taejin Park <[email protected]>

* Style fix and isort

Signed-off-by: Taejin Park <[email protected]>

* jbalam-nv comments reflected

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected comments and created _diar_label.py

Signed-off-by: Taejin Park <[email protected]>

* Typo fix and style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed target_spks[0] index error

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* LGTM unused import IterDataset

Signed-off-by: Taejin Park <[email protected]>

* revert collection doc year

Signed-off-by: Taejin Park <[email protected]>

* Code format error in collections.py

Signed-off-by: Taejin Park <[email protected]>

* fix collections space format error

Signed-off-by: Taejin Park <[email protected]>

* merged main correctly

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected all comments and tested

Signed-off-by: Taejin Park <[email protected]>

* style fix and LGTM

Signed-off-by: Taejin Park <[email protected]>

* rttm_filepath to rttm_file and removed self included funcs, tested

Signed-off-by: Taejin Park <[email protected]>

Co-authored-by: Nithin Rao <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* removed references to data_dir

Signed-off-by: Matvei Novikov <[email protected]>

* added missing parameters to data preparation script

Signed-off-by: Matvei Novikov <[email protected]>

* removed unnecessary file extension check

Signed-off-by: Matvei Novikov <[email protected]>

* Add ASR CTC Decoding module (#4342)

* Initial commit

Signed-off-by: smajumdar <[email protected]>

* Full support for decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Temp

Signed-off-by: smajumdar <[email protected]>

* Fix labels of y_sequence

Signed-off-by: smajumdar <[email protected]>

* Set support for sentencepiece subword merging

Signed-off-by: smajumdar <[email protected]>

* Fix char and word based token merge alignment

Signed-off-by: smajumdar <[email protected]>

* Revert incorrect change

Signed-off-by: smajumdar <[email protected]>

* Update docstring

Signed-off-by: smajumdar <[email protected]>

* Improve compatibility with greedy tokens and log probs

Signed-off-by: smajumdar <[email protected]>

* Update scripts to use decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix decoding of ctc models

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Option to disable mp in VAD via num_workers=1 (#4317)

* Option to disable mp in VAD via num_workers=1

In certain environments python multiprocessing can deadlock. This adds a convenient version to disable by setting num_workers to 1.

Signed-off-by: Georg Kucsko <[email protected]>

* add none handling

Signed-off-by: Georg Kucsko <[email protected]>

* additional none handling

Signed-off-by: Georg Kucsko <[email protected]>

Co-authored-by: fayejf <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* remove redundant bias expand (#4382)

* remove redundant bias expand

Signed-off-by: Xiaowei Ren <[email protected]>

* delete redundant code

Signed-off-by: Xiaowei Ren <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* fixed style

Signed-off-by: Matvei Novikov <[email protected]>

* Add option for specifying wandb save_dir from config (#4379)

* give option to user to specify wandb save dir via config

Signed-off-by: Shantanu Acharya <[email protected]>

* create save_dir directory for wandb logger if not exists

Signed-off-by: Shantanu Acharya <[email protected]>

* update save_dir get method with a default value

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* [Bugfix][TTS] wrong order of returned tuple for general_collate_fn. (#4388)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Merge r1.10.0 main (#4398)

* update branch

Signed-off-by: ericharper <[email protected]>

* Set headscale false (#4364)

Signed-off-by: MaximumEntropy <[email protected]>

* Add wandb as dependency (#4365)

Signed-off-by: smajumdar <[email protected]>

* Raise trainer error (#4356)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Set headscale false (#4364) (#4366)

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: smajumdar <[email protected]>

* Finetuning changes for BART (#4003)

* Temp

Signed-off-by: MaximumEntropy <[email protected]>

* Checkpoint converter to nemo for bart

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Make position embedding expansion specific to a batch to avoid checkpoint size mismatches (#4357)

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Fix logging warning

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Fix electronic bug, new time ITN rule (#4355)

* fix electronic bug

Signed-off-by: ekmb <[email protected]>

* add new itn time rule

Signed-off-by: ekmb <[email protected]>

* revert domain changes

Signed-off-by: ekmb <[email protected]>

* remove repetition

Signed-off-by: ekmb <[email protected]>

* Correct support for dataclasses in default module dim (#4372)

* Correct support for dataclasses in default module dim

Signed-off-by: smajumdar <[email protected]>

* Fix path for save of results

Signed-off-by: smajumdar <[email protected]>

* fix pad id bug (#4377)

Signed-off-by: Yi Dong <[email protected]>

* Question answering bug fix (#4381)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assistant data processsor

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins for SGDGEN local checkpoint

Signed-off-by: Zhilin Wang <[email protected]>

* update style

Signed-off-by: Zhilin Wang <[email protected]>

* use local vocab file for Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* patch for Jenkins CI using local file

Signed-off-by: Zhilin Wang <[email protected]>

* add slot filling prediction and metrics

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused code

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* refactor metrics code out of Dialogue GPT Model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate backward compatible support for IntentSlotClassificationModel (bert model)

Signed-off-by: Zhilin Wang <[email protected]>

* save prediction file for IntentSlotClassification

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue gpt model training for megatron gpt

Signed-off-by: Zhilin Wang <[email protected]>

* remove batch generate for HF GPT2, which causes lower performance

Signed-off-by: Zhilin Wang <[email protected]>

* add few shot capability to dialogue gpt model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile and remove unused import

Signed-off-by: Zhilin Wang <[email protected]>

* update code description and clarity

Signed-off-by: Zhilin Wang <[email protected]>

* address PR comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate compatibility with ZeroShotIntentModel

Signed-off-by: Zhilin Wang <[email protected]>

* rename folder to dialogue due to increased scope and further refactor for clarity

Signed-off-by: Zhilin Wang <[email protected]>

* added dialogue GPT for sequence generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* add CI test for DialogueGPTGenerationModel

Signed-off-by: Zhilin Wang <[email protected]>

* integrate DialogueS2SGenerationModel for generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* modify huggingface utils to support HF t5/BART models

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* fix bleu metric style

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* update based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 2 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 3 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* integrate sgd generation based on user user utterance and system slot-values to generate system utterance

Signed-off-by: Zhilin Wang <[email protected]>

* add validation model saving capabilities

Signed-off-by: Zhilin Wang <[email protected]>

* cleaned up code for SGD Based Answer extender

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue Generation CI

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* fix Jenkins CI issue"

Signed-off-by: Zhilin Wang <[email protected]>

* add support for design dataset

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary imports

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support megatron for dialogue_s2s_generation_model

Signed-off-by: Zhilin Wang <[email protected]>

* reduce loaded samples in MSMarcoDataProcessor to 64 when cfg.model.dataset.debug_mode=True

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update CI

Signed-off-by: Zhilin Wang <[email protected]>

* update checkpoint and predictions filename to include epoch number

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate HF BART MNLI into zero shot intent model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Nearest Neighbour Model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* refactor Dialogue SGD Data Processor to make interface for models cleaner

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue S2S Generation model for DialogueSGDDataProcessor interface

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support sgd and drive thru datasets by zero shot model and nearest neighbour model

Signed-off-by: Zhilin Wang <[email protected]>

* add prediction saving code to nearest neighbour and zero shot intent models

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo in sgd data processor

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Mellon QA Data Processor

Signed-off-by: Zhilin Wang <[email protected]>

* update mellon qa

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue.py to remove outdated info

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* add dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* address review comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix for cfg

Signed-off-by: Zhilin Wang <[email protected]>

* make dependency on apex optional

Signed-off-by: Zhilin Wang <[email protected]>

* change NLPDDPluggin calling logic to make it possible to run without apex

Signed-off-by: Zhilin Wang <[email protected]>

* add first draft of tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* reduce ms marco size by removing lines without wellFormedAnswers

Signed-off-by: Zhilin Wang <[email protected]>

* address pr comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update colab tutorial link in dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* include unit test and some refactor to facilitate unit test

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* address pr issues

Signed-off-by: Zhilin Wang <[email protected]>

* remove typos in dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* support larger files for question answering

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary artifacts to reduce memory use

Signed-off-by: Zhilin Wang <[email protected]>

* put 0 tensor to device

Signed-off-by: Zhilin Wang <[email protected]>

* update link within dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously delete files

Signed-off-by: Zhilin Wang <[email protected]>

* update error handling when loss = nan

Signed-off-by: Zhilin Wang <[email protected]>

* update nan handling

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss func

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss

Signed-off-by: Zhilin Wang <[email protected]>

* fix type error raised in qa_dataset.py

Signed-off-by: Zhilin Wang <[email protected]>

* add error checking message

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update exp logging

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* limit number of negative samples

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused methods and style fix

Signed-off-by: Zhilin Wang <[email protected]>

* add more documentation

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* changes base on PR review

Signed-off-by: Zhilin Wang <[email protected]>

* set wandb logger falseby default

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

* style fix

* correct typo

* style fix

* style fix

Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>

* Fix ASR Typos in tutorials (#4384)

* Fix typos

Signed-off-by: smajumdar <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
(cherry picked from commit 0322b158f26a0b690edca7a84714e33752283923)

Co-authored-by: Travis Bartley <[email protected]>

* Add Docs for NeMo Adapters (#4369)

Signed-off-by: smajumdar <[email protected]>

* Update NeMo docs (#4397)

Signed-off-by: smajumdar <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* update branch

Signed-off-by: ericharper <[email protected]>

* remove Copy of

Signed-off-by: ericharper <[email protected]>

Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Micha Livne <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Travis Bartley <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* [bugfix][TTS] pitch, voiced_mask, prob_voiced have the same values. (#4392)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Fixing import error in some cases (#4401)

Signed-off-by: Boris Fomitchev <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Fixing bugs in calling method ctc_decoder_predictions_tensor. (#4414)

* updated ctc decoding calls.

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Update with new conformer checkpoints. (#4417)

Signed-off-by: Matvei Novikov <[email protected]>

* [TTS] add static method decorator. (#4443)

* [TTS] add static method decorator.

Signed-off-by: Xuesong Yang <[email protected]>

* remove protect prefix

Signed-off-by: Xuesong Yang <[email protected]>

* fixed style error

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

Co-authored-by: Taejin Park <[email protected]>
Co-authored-by: Nithin Rao <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Georg Kucsko <[email protected]>
Co-authored-by: fayejf <[email protected]>
Co-authored-by: Xiaowei Ren <[email protected]>
Co-authored-by: Shantanu Acharya <[email protected]>
Co-authored-by: Travis Bartley <[email protected]>
Co-authored-by: tbartley94 <[email protected]>
Co-authored-by: Xuesong Yang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Micha Livne <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Boris Fomitchev <[email protected]>
Co-authored-by: Vahid Noroozi <[email protected]>

* bug fix - sample rate was being ignored in vocoder dataset when not loading mel

Signed-off-by: Paarth Neekhara <[email protected]>

* Add ITN pt (#4516)

* Add ITN pt

Signed-off-by: Guilherme Steinmann <[email protected]>

* Fix style

Signed-off-by: Guilherme Steinmann <[email protected]>

* Fix style

Signed-off-by: Guilherme Steinmann <[email protected]>

* Update copyright year to 2022 on ITN pt rules and tests

Signed-off-by: Guilherme Steinmann <[email protected]>

* Fixed WER initialization in ASR_with_Nemo notebook (#4523)

Signed-off-by: Ante Jukić <[email protected]>

Co-authored-by: Ante Jukić <[email protected]>

* Update cmudict (#4510)

phoneme IY1 -> IH1 in NVIDIA
Added phonemes for CUSTOMIZABLE

Update cmudict file revision and its reference.

Signed-off-by: Jason Roche <[email protected]>

Co-authored-by: Jason Roche <[email protected]>
Co-authored-by: Xuesong Yang <[email protected]>

* [Add] Support for Different LRs with Param Groups (#4508)

* add support for param groups

Signed-off-by: stevehuang52 <[email protected]>

* make config more general

Signed-off-by: stevehuang52 <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* Weighted bucketing (#4474)

* Add silence handling for speaker diarization pipeline (#4512)

* initial commit

Signed-off-by: nithinraok <[email protected]>

* fixed silence wav file issue causing clustering to evaluate on null embeddings

Signed-off-by: nithinraok <[email protected]>

* fixed zero duration issue

Signed-off-by: nithinraok <[email protected]>

* updated with comments

Signed-off-by: nithinraok <[email protected]>

* minor doc change

Signed-off-by: nithinraok <[email protected]>

* update log

Signed-off-by: nithinraok <[email protected]>

* Fix runtime check (#4501)

* Runtime check refinements

Signed-off-by: Boris Fomitchev <[email protected]>

* Added fp32 casting for ASR nets export

Signed-off-by: Boris Fomitchev <[email protected]>

* style

Signed-off-by: Boris Fomitchev <[email protected]>

* Used torch.float32 for clarity

Signed-off-by: Boris Fomitchev <[email protected]>

* Fixing parameters passing

Signed-off-by: Boris Fomitchev <[email protected]>

* Update finetune label models (#4504)

* initial_script

Signed-off-by: nithinraok <[email protected]>

* move old script

Signed-off-by: nithinraok <[email protected]>

* remove finetune func from label models

Signed-off-by: nithinraok <[email protected]>

* style clean

Signed-off-by: nithinraok <[email protected]>

* updated config

Signed-off-by: nithinraok <[email protected]>

* update tutorial

Signed-off-by: nithinraok <[email protected]>

* lgtm fixes

Signed-off-by: nithinraok <[email protected]>

* updated based on comments

Signed-off-by: nithinraok <[email protected]>

* update doc

Signed-off-by: nithinraok <[email protected]>

* [ASR][Breaking Change] Update signature of Hypothesis alignments (#4511)

* Preserve logprobs when preserving alignments

Signed-off-by: smajumdar <[email protected]>

* Update tests for rnnt gredy and beam search

Signed-off-by: smajumdar <[email protected]>

* Update all dependents of alignments

Signed-off-by: smajumdar <[email protected]>

* Update docs

Signed-off-by: smajumdar <[email protected]>

* Weighted bucketing (#4530)

* Additional sentencepiece args - Byte fallback, split digits, split_on_whitespace (#4525)

* Fix geglu without fusion

Signed-off-by: MaximumEntropy <[email protected]>

* Add extra args

Signed-off-by: MaximumEntropy <[email protected]>

* Reset transformer

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Fix spm arg

Signed-off-by: MaximumEntropy <[email protected]>

* Fix help string

Signed-off-by: MaximumEntropy <[email protected]>

* Add support for ASR Adapter Auxiliary Losses (#4480)

* Add support for access mixin registry of custom losses

Signed-off-by: smajumdar <[email protected]>

* add support for asr custom losses

Signed-off-by: smajumdar <[email protected]>

* Update for l2 loss

Signed-off-by: smajumdar <[email protected]>

* Add unittests

Signed-off-by: smajumdar <[email protected]>

* Add unittests

Signed-off-by: smajumdar <[email protected]>

* Add unittests

Signed-off-by: smajumdar <[email protected]>

* Update registration of tensors to reset after finishing step

Signed-off-by: smajumdar <[email protected]>

* Remove comment

Signed-off-by: smajumdar <[email protected]>

* Remove comment

Signed-off-by: smajumdar <[email protected]>

* Update SSL models

Signed-off-by: smajumdar <[email protected]>

* Add support for validation step properly registering tensors

Signed-off-by: smajumdar <[email protected]>

* Move reset of registry outside

Signed-off-by: smajumdar <[email protected]>

* update (#4520)

Signed-off-by: stevehuang52 <[email protected]>

* fix duplex inference with grammars (#4517)

* fix duplex inference with grammars

Signed-off-by: ekmb <[email protected]>

* add ci test for duplex, fix electronic last sym bug

Signed-off-by: ekmb <[email protected]>

* test fix

Signed-off-by: ekmb <[email protected]>

* fix jenkins

Signed-off-by: ekmb <[email protected]>

* update jenkins grammars

Signed-off-by: ekmb <[email protected]>

* add pt to the docs

Signed-off-by: ekmb <[email protected]>

* fix jenkins

Signed-off-by: ekmb <[email protected]>

* disable test

Signed-off-by: ekmb <[email protected]>

* fix jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins refactor

Signed-off-by: ekmb <[email protected]>

* fix jenkins

Signed-off-by: ekmb <[email protected]>

* fix jenkins

Signed-off-by: ekmb <[email protected]>

* fix jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins

Signed-off-by: ekmb <[email protected]>

* test

Signed-off-by: ekmb <[email protected]>

* test

Signed-off-by: ekmb <[email protected]>

* test

Signed-off-by: ekmb <[email protected]>

* test

Signed-off-by: ekmb <[email protected]>

Co-authored-by: Yang Zhang <[email protected]>

* Add Bucketing support to TarredAudioToClassificationLabelDataset (#4465)

* Add Bucketing support to TarredAudioToClassificationLabelDataset

Signed-off-by: Ewald Enzinger <[email protected]>

* Add MTEncDec Finetune support (#4540)

* add FT support

Signed-off-by: Abhinav Khattar <[email protected]>

* rm preproc

Signed-off-by: Abhinav Khattar <[email protected]>

* review changes

Signed-off-by: Abhinav Khattar <[email protected]>

* add CI

Signed-off-by: Abhinav Khattar <[email protected]>

* newline fix

Signed-off-by: Abhinav Khattar <[email protected]>

* CI fix

Signed-off-by: Abhinav Khattar <[email protected]>

* clean up

Signed-off-by: Abhinav Khattar <[email protected]>

* post training cleanup

Signed-off-by: Abhinav Khattar <[email protected]>

* test

Signed-off-by: Abhinav Khattar <[email protected]>

* revert

Signed-off-by: Abhinav Khattar <[email protected]>

* CI test

Signed-off-by: Abhinav Khattar <[email protected]>

* revert CI changes

Signed-off-by: Abhinav Khattar <[email protected]>

* original CI

Signed-off-by: Abhinav Khattar <[email protected]>

Co-authored-by: Sandeep Subramanian <[email protected]>

* Add nsys profiling (#4539)

* add nsys profiling

Signed-off-by: ericharper <[email protected]>

* only access omegaconf in setup

Signed-off-by: ericharper <[email protected]>

* use robust get_rank function

Signed-off-by: ericharper <[email protected]>

* simplify

Signed-off-by: ericharper <[email protected]>

* Update megatron prompt learning interface to dialogue  (#4545)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assistant data …
paarthneekhara added a commit to paarthneekhara/NeMo that referenced this pull request Jul 24, 2022
* Megatron BART BOS / EOS bug fix (#4495)

* 1. Debugging.

Signed-off-by: Micha Livne <[email protected]>

* 1. BART dataset fixes missing <EOS> for deocder output.

Signed-off-by: Micha Livne <[email protected]>

* 1. Debugging.

Signed-off-by: Micha Livne <[email protected]>

* 1. Debugging.

Signed-off-by: Micha Livne <[email protected]>

* 1. Removed extra padding from BARTDataset.

Signed-off-by: Micha Livne <[email protected]>

* GPT Prompt Learning Improvements (#4496)

* Updated pipeline parallel code to speed up training

Signed-off-by: Virginia Adams <[email protected]>

* Load global batch size not local mini batch size

Signed-off-by: Virginia Adams <[email protected]>

* Python reformatting

Signed-off-by: Virginia Adams <[email protected]>

* Megatron perceiver with tensor parallelism only (#4318)

* Temp

Signed-off-by: MaximumEntropy <[email protected]>

* Add megatron dataset

Signed-off-by: MaximumEntropy <[email protected]>

* Update config and fix global batch fetcher

Signed-off-by: MaximumEntropy <[email protected]>

* Add dataset class

Signed-off-by: MaximumEntropy <[email protected]>

* Update comments

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Update yaml

Signed-off-by: MaximumEntropy <[email protected]>

* Fix duplicate yaml key

Signed-off-by: MaximumEntropy <[email protected]>

* Translate method and preprocess script for raw text

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Remove pdb

Signed-off-by: MaximumEntropy <[email protected]>

* Fix arg name

Signed-off-by: MaximumEntropy <[email protected]>

* Fix other arg

Signed-off-by: MaximumEntropy <[email protected]>

* Change sampler back

Signed-off-by: MaximumEntropy <[email protected]>

* Move back to global batch fetcher to use distributed sampler

Signed-off-by: MaximumEntropy <[email protected]>

* Add text memmap data

Signed-off-by: MaximumEntropy <[email protected]>

* Update monitor

Signed-off-by: MaximumEntropy <[email protected]>

* Fixes for PP

Signed-off-by: MaximumEntropy <[email protected]>

* Remove unused import

Signed-off-by: MaximumEntropy <[email protected]>

* Truncate examples in text memmap

Signed-off-by: MaximumEntropy <[email protected]>

* NMT training batch interpolation key

Signed-off-by: MaximumEntropy <[email protected]>

* tarred data fix

Signed-off-by: MaximumEntropy <[email protected]>

* Change dataset type check

Signed-off-by: MaximumEntropy <[email protected]>

* Fix sampler

Signed-off-by: MaximumEntropy <[email protected]>

* Pass dataset cfg to determine type

Signed-off-by: MaximumEntropy <[email protected]>

* Log global step on validation step as well

Signed-off-by: MaximumEntropy <[email protected]>

* Fix NMT model saving with artifacts

Signed-off-by: MaximumEntropy <[email protected]>

* Initialize DDP in decode if not initialized. Needed for inference only mode

Signed-off-by: MaximumEntropy <[email protected]>

* Megatron NMT inference script

Signed-off-by: MaximumEntropy <[email protected]>

* Inference config file

Signed-off-by: MaximumEntropy <[email protected]>

* hardcode max delta temporarily

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* detokenizer if processor is not none

Signed-off-by: MaximumEntropy <[email protected]>

* Sampler config

Signed-off-by: MaximumEntropy <[email protected]>

* Compat with configs without sampler arg

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Comment for validation dataset type

Signed-off-by: MaximumEntropy <[email protected]>

* Fix tokenizer building

Signed-off-by: MaximumEntropy <[email protected]>

* CI test for megatron nmt

Signed-off-by: MaximumEntropy <[email protected]>

* Fix tokenizer in restore

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* O2 restore from fix

Signed-off-by: MaximumEntropy <[email protected]>

* Remove print

Signed-off-by: MaximumEntropy <[email protected]>

* Change tokenizer model name in config

Signed-off-by: MaximumEntropy <[email protected]>

* Logging

Signed-off-by: MaximumEntropy <[email protected]>

* Set seed for distributed sampler

Signed-off-by: MaximumEntropy <[email protected]>

* Cluster debugging messages

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Fix max generation delta

Signed-off-by: MaximumEntropy <[email protected]>

* No LM Init

Signed-off-by: MaximumEntropy <[email protected]>

* Use nlp save restore connector

Signed-off-by: MaximumEntropy <[email protected]>

* Remove useless infer args

Signed-off-by: MaximumEntropy <[email protected]>

* Typo

Signed-off-by: MaximumEntropy <[email protected]>

* UTF8 safe print of translation result

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Add save restore connector back with comment

Signed-off-by: MaximumEntropy <[email protected]>

* Refactor

Signed-off-by: MaximumEntropy <[email protected]>

* Fix CI test

Signed-off-by: MaximumEntropy <[email protected]>

* Add missing args

Signed-off-by: MaximumEntropy <[email protected]>

* Address comments

Signed-off-by: MaximumEntropy <[email protected]>

* Empty to restart

* Fix CI test

Signed-off-by: MaximumEntropy <[email protected]>

* Check for test ds

Signed-off-by: MaximumEntropy <[email protected]>

* set fusion to false

Signed-off-by: MaximumEntropy <[email protected]>

* Initial perceiver encoder

Signed-off-by: MaximumEntropy <[email protected]>

* Perceiver with PP=1

Signed-off-by: MaximumEntropy <[email protected]>

* Remove init cross attn

Signed-off-by: MaximumEntropy <[email protected]>

* CI test and remove init cross attn arg

Signed-off-by: MaximumEntropy <[email protected]>

* Remove init cross attn layers from file

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Clean up

Signed-off-by: MaximumEntropy <[email protected]>

* update branch

Signed-off-by: ericharper <[email protected]>

* Set headscale false (#4364)

Signed-off-by: MaximumEntropy <[email protected]>

* Add wandb as dependency (#4365)

Signed-off-by: smajumdar <[email protected]>

* Raise trainer error (#4356)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Set headscale false (#4364) (#4366)

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: smajumdar <[email protected]>

* Finetuning changes for BART (#4003)

* Temp

Signed-off-by: MaximumEntropy <[email protected]>

* Checkpoint converter to nemo for bart

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Make position embedding expansion specific to a batch to avoid checkpoint size mismatches (#4357)

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Fix logging warning

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Refactor bias act fusion

Signed-off-by: MaximumEntropy <[email protected]>

* Update NMT config

Signed-off-by: MaximumEntropy <[email protected]>

* Fix electronic bug, new time ITN rule (#4355)

* fix electronic bug

Signed-off-by: ekmb <[email protected]>

* add new itn time rule

Signed-off-by: ekmb <[email protected]>

* revert domain changes

Signed-off-by: ekmb <[email protected]>

* remove repetition

Signed-off-by: ekmb <[email protected]>

* Update ci tests

Signed-off-by: MaximumEntropy <[email protected]>

* Correct support for dataclasses in default module dim (#4372)

* Correct support for dataclasses in default module dim

Signed-off-by: smajumdar <[email protected]>

* Fix path for save of results

Signed-off-by: smajumdar <[email protected]>

* fix pad id bug (#4377)

Signed-off-by: Yi Dong <[email protected]>

* Question answering bug fix (#4381)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assistant data processsor

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins for SGDGEN local checkpoint

Signed-off-by: Zhilin Wang <[email protected]>

* update style

Signed-off-by: Zhilin Wang <[email protected]>

* use local vocab file for Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* patch for Jenkins CI using local file

Signed-off-by: Zhilin Wang <[email protected]>

* add slot filling prediction and metrics

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused code

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* refactor metrics code out of Dialogue GPT Model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate backward compatible support for IntentSlotClassificationModel (bert model)

Signed-off-by: Zhilin Wang <[email protected]>

* save prediction file for IntentSlotClassification

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue gpt model training for megatron gpt

Signed-off-by: Zhilin Wang <[email protected]>

* remove batch generate for HF GPT2, which causes lower performance

Signed-off-by: Zhilin Wang <[email protected]>

* add few shot capability to dialogue gpt model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile and remove unused import

Signed-off-by: Zhilin Wang <[email protected]>

* update code description and clarity

Signed-off-by: Zhilin Wang <[email protected]>

* address PR comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate compatibility with ZeroShotIntentModel

Signed-off-by: Zhilin Wang <[email protected]>

* rename folder to dialogue due to increased scope and further refactor for clarity

Signed-off-by: Zhilin Wang <[email protected]>

* added dialogue GPT for sequence generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* add CI test for DialogueGPTGenerationModel

Signed-off-by: Zhilin Wang <[email protected]>

* integrate DialogueS2SGenerationModel for generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* modify huggingface utils to support HF t5/BART models

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* fix bleu metric style

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* update based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 2 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 3 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* integrate sgd generation based on user user utterance and system slot-values to generate system utterance

Signed-off-by: Zhilin Wang <[email protected]>

* add validation model saving capabilities

Signed-off-by: Zhilin Wang <[email protected]>

* cleaned up code for SGD Based Answer extender

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue Generation CI

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* fix Jenkins CI issue"

Signed-off-by: Zhilin Wang <[email protected]>

* add support for design dataset

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary imports

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support megatron for dialogue_s2s_generation_model

Signed-off-by: Zhilin Wang <[email protected]>

* reduce loaded samples in MSMarcoDataProcessor to 64 when cfg.model.dataset.debug_mode=True

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update CI

Signed-off-by: Zhilin Wang <[email protected]>

* update checkpoint and predictions filename to include epoch number

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate HF BART MNLI into zero shot intent model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Nearest Neighbour Model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* refactor Dialogue SGD Data Processor to make interface for models cleaner

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue S2S Generation model for DialogueSGDDataProcessor interface

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support sgd and drive thru datasets by zero shot model and nearest neighbour model

Signed-off-by: Zhilin Wang <[email protected]>

* add prediction saving code to nearest neighbour and zero shot intent models

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo in sgd data processor

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Mellon QA Data Processor

Signed-off-by: Zhilin Wang <[email protected]>

* update mellon qa

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue.py to remove outdated info

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* add dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* address review comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix for cfg

Signed-off-by: Zhilin Wang <[email protected]>

* make dependency on apex optional

Signed-off-by: Zhilin Wang <[email protected]>

* change NLPDDPluggin calling logic to make it possible to run without apex

Signed-off-by: Zhilin Wang <[email protected]>

* add first draft of tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* reduce ms marco size by removing lines without wellFormedAnswers

Signed-off-by: Zhilin Wang <[email protected]>

* address pr comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update colab tutorial link in dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* include unit test and some refactor to facilitate unit test

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* address pr issues

Signed-off-by: Zhilin Wang <[email protected]>

* remove typos in dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* support larger files for question answering

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary artifacts to reduce memory use

Signed-off-by: Zhilin Wang <[email protected]>

* put 0 tensor to device

Signed-off-by: Zhilin Wang <[email protected]>

* update link within dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously delete files

Signed-off-by: Zhilin Wang <[email protected]>

* update error handling when loss = nan

Signed-off-by: Zhilin Wang <[email protected]>

* update nan handling

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss func

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss

Signed-off-by: Zhilin Wang <[email protected]>

* fix type error raised in qa_dataset.py

Signed-off-by: Zhilin Wang <[email protected]>

* add error checking message

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update exp logging

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* limit number of negative samples

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused methods and style fix

Signed-off-by: Zhilin Wang <[email protected]>

* add more documentation

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* changes base on PR review

Signed-off-by: Zhilin Wang <[email protected]>

* set wandb logger falseby default

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

* style fix

* correct typo

* style fix

* style fix

Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>

* Fix ASR Typos in tutorials (#4384)

* Fix typos

Signed-off-by: smajumdar <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
(cherry picked from commit 0322b158f26a0b690edca7a84714e33752283923)

Co-authored-by: Travis Bartley <[email protected]>

* Add Docs for NeMo Adapters (#4369)

Signed-off-by: smajumdar <[email protected]>

* Update NeMo docs (#4397)

Signed-off-by: smajumdar <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* Punctuation and capitalization tests race condition (#4399)

* Add draft of race condition fixes

Signed-off-by: PeganovAnton <[email protected]>

* Minor improvements

Signed-off-by: PeganovAnton <[email protected]>

* More race condition fixes

Signed-off-by: PeganovAnton <[email protected]>

* Improve error message

Signed-off-by: PeganovAnton <[email protected]>

* Improve error message

Signed-off-by: PeganovAnton <[email protected]>

* Improve error message

Signed-off-by: PeganovAnton <[email protected]>

* bias act fusion changes

Signed-off-by: MaximumEntropy <[email protected]>

* Address comments

Signed-off-by: MaximumEntropy <[email protected]>

* Fix geglu without fusion

Signed-off-by: MaximumEntropy <[email protected]>

* Reset files to main

Signed-off-by: MaximumEntropy <[email protected]>

* Remove hidden blocks

Signed-off-by: MaximumEntropy <[email protected]>

* Fix style

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>
Co-authored-by: Abhinav Khattar <[email protected]>
Co-authored-by: ericharper <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Travis Bartley <[email protected]>
Co-authored-by: PeganovAnton <[email protected]>

* NMESC speaker counting algorithm update (#4500)

* initial commit

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Default maj_vote = False, max_rp=0.25

Signed-off-by: Taejin Park <[email protected]>

* doc strings and style fix

Signed-off-by: Taejin Park <[email protected]>

* Docstring minor edit

Signed-off-by: Taejin Park <[email protected]>

* Default False in the functions

Signed-off-by: Taejin Park <[email protected]>

* fixed repeated variable

Signed-off-by: Taejin Park <[email protected]>

* Default as maj_vote=False

Signed-off-by: Taejin Park <[email protected]>

* removed redundant part in wrtie_rttm func

Signed-off-by: Taejin Park <[email protected]>

* Removed unused function

Signed-off-by: Taejin Park <[email protected]>

* Updated and tested silence and very short samples

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Style fix and removing unnecessary parts

Signed-off-by: Taejin Park <[email protected]>

* unused variables are removed

Signed-off-by: Taejin Park <[email protected]>

* Fixed commented torch.jit.script

Signed-off-by: Taejin Park <[email protected]>

* majority voting update

Signed-off-by: Taejin Park <[email protected]>

* cancelling the update on speaker_utils and clus_diarizer

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* bug fix

Signed-off-by: Taejin Park <[email protected]>

* Added fp32 converting for torch.mm

Signed-off-by: Taejin Park <[email protected]>

Co-authored-by: Nithin Rao <[email protected]>

* Fix dataset parameter typo on tacotron2 example yaml (#4471)

Signed-off-by: saarus72 <[email protected]>

Co-authored-by: Xuesong Yang <[email protected]>

* Noam lr sched: do not force min_lr after max_steps (#4472)

Signed-off-by: Adrian Lancucki <[email protected]>

Co-authored-by: Adrian Lancucki <[email protected]>
Co-authored-by: Xuesong Yang <[email protected]>

* Refactor for punctuation model (#4367)

* Dataloader, collector, loss and metric for multiscale diarization decoder  (#4187)

* First commit

Signed-off-by: Taejin Park <[email protected]>

* Checked funtionality and imports

Signed-off-by: Taejin Park <[email protected]>

* fixed import issues

Signed-off-by: Taejin Park <[email protected]>

* Removed the changed made by mistake

Signed-off-by: Taejin Park <[email protected]>

* Style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM errors 001

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM and style fix

Signed-off-by: Taejin Park <[email protected]>

* Changed docstrings

Signed-off-by: Taejin Park <[email protected]>

* LGTM again

Signed-off-by: Taejin Park <[email protected]>

* Removed unnecessary torch setting lines

Signed-off-by: Taejin Park <[email protected]>

* Style fix and isort

Signed-off-by: Taejin Park <[email protected]>

* jbalam-nv comments reflected

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected comments and created _diar_label.py

Signed-off-by: Taejin Park <[email protected]>

* Typo fix and style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed target_spks[0] index error

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* LGTM unused import IterDataset

Signed-off-by: Taejin Park <[email protected]>

* revert collection doc year

Signed-off-by: Taejin Park <[email protected]>

* Code format error in collections.py

Signed-off-by: Taejin Park <[email protected]>

* fix collections space format error

Signed-off-by: Taejin Park <[email protected]>

* merged main correctly

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected all comments and tested

Signed-off-by: Taejin Park <[email protected]>

* style fix and LGTM

Signed-off-by: Taejin Park <[email protected]>

* rttm_filepath to rttm_file and removed self included funcs, tested

Signed-off-by: Taejin Park <[email protected]>

Co-authored-by: Nithin Rao <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* removed references to data_dir

Signed-off-by: Matvei Novikov <[email protected]>

* added missing parameters to data preparation script

Signed-off-by: Matvei Novikov <[email protected]>

* removed unnecessary file extension check

Signed-off-by: Matvei Novikov <[email protected]>

* Add ASR CTC Decoding module (#4342)

* Initial commit

Signed-off-by: smajumdar <[email protected]>

* Full support for decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Temp

Signed-off-by: smajumdar <[email protected]>

* Fix labels of y_sequence

Signed-off-by: smajumdar <[email protected]>

* Set support for sentencepiece subword merging

Signed-off-by: smajumdar <[email protected]>

* Fix char and word based token merge alignment

Signed-off-by: smajumdar <[email protected]>

* Revert incorrect change

Signed-off-by: smajumdar <[email protected]>

* Update docstring

Signed-off-by: smajumdar <[email protected]>

* Improve compatibility with greedy tokens and log probs

Signed-off-by: smajumdar <[email protected]>

* Update scripts to use decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix decoding of ctc models

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Option to disable mp in VAD via num_workers=1 (#4317)

* Option to disable mp in VAD via num_workers=1

In certain environments python multiprocessing can deadlock. This adds a convenient version to disable by setting num_workers to 1.

Signed-off-by: Georg Kucsko <[email protected]>

* add none handling

Signed-off-by: Georg Kucsko <[email protected]>

* additional none handling

Signed-off-by: Georg Kucsko <[email protected]>

Co-authored-by: fayejf <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* remove redundant bias expand (#4382)

* remove redundant bias expand

Signed-off-by: Xiaowei Ren <[email protected]>

* delete redundant code

Signed-off-by: Xiaowei Ren <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* fixed style

Signed-off-by: Matvei Novikov <[email protected]>

* Add option for specifying wandb save_dir from config (#4379)

* give option to user to specify wandb save dir via config

Signed-off-by: Shantanu Acharya <[email protected]>

* create save_dir directory for wandb logger if not exists

Signed-off-by: Shantanu Acharya <[email protected]>

* update save_dir get method with a default value

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* [Bugfix][TTS] wrong order of returned tuple for general_collate_fn. (#4388)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Merge r1.10.0 main (#4398)

* update branch

Signed-off-by: ericharper <[email protected]>

* Set headscale false (#4364)

Signed-off-by: MaximumEntropy <[email protected]>

* Add wandb as dependency (#4365)

Signed-off-by: smajumdar <[email protected]>

* Raise trainer error (#4356)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Set headscale false (#4364) (#4366)

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: smajumdar <[email protected]>

* Finetuning changes for BART (#4003)

* Temp

Signed-off-by: MaximumEntropy <[email protected]>

* Checkpoint converter to nemo for bart

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Make position embedding expansion specific to a batch to avoid checkpoint size mismatches (#4357)

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Fix logging warning

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Fix electronic bug, new time ITN rule (#4355)

* fix electronic bug

Signed-off-by: ekmb <[email protected]>

* add new itn time rule

Signed-off-by: ekmb <[email protected]>

* revert domain changes

Signed-off-by: ekmb <[email protected]>

* remove repetition

Signed-off-by: ekmb <[email protected]>

* Correct support for dataclasses in default module dim (#4372)

* Correct support for dataclasses in default module dim

Signed-off-by: smajumdar <[email protected]>

* Fix path for save of results

Signed-off-by: smajumdar <[email protected]>

* fix pad id bug (#4377)

Signed-off-by: Yi Dong <[email protected]>

* Question answering bug fix (#4381)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assistant data processsor

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins for SGDGEN local checkpoint

Signed-off-by: Zhilin Wang <[email protected]>

* update style

Signed-off-by: Zhilin Wang <[email protected]>

* use local vocab file for Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* patch for Jenkins CI using local file

Signed-off-by: Zhilin Wang <[email protected]>

* add slot filling prediction and metrics

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused code

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* refactor metrics code out of Dialogue GPT Model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate backward compatible support for IntentSlotClassificationModel (bert model)

Signed-off-by: Zhilin Wang <[email protected]>

* save prediction file for IntentSlotClassification

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue gpt model training for megatron gpt

Signed-off-by: Zhilin Wang <[email protected]>

* remove batch generate for HF GPT2, which causes lower performance

Signed-off-by: Zhilin Wang <[email protected]>

* add few shot capability to dialogue gpt model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile and remove unused import

Signed-off-by: Zhilin Wang <[email protected]>

* update code description and clarity

Signed-off-by: Zhilin Wang <[email protected]>

* address PR comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate compatibility with ZeroShotIntentModel

Signed-off-by: Zhilin Wang <[email protected]>

* rename folder to dialogue due to increased scope and further refactor for clarity

Signed-off-by: Zhilin Wang <[email protected]>

* added dialogue GPT for sequence generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* add CI test for DialogueGPTGenerationModel

Signed-off-by: Zhilin Wang <[email protected]>

* integrate DialogueS2SGenerationModel for generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* modify huggingface utils to support HF t5/BART models

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* fix bleu metric style

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* update based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 2 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 3 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* integrate sgd generation based on user user utterance and system slot-values to generate system utterance

Signed-off-by: Zhilin Wang <[email protected]>

* add validation model saving capabilities

Signed-off-by: Zhilin Wang <[email protected]>

* cleaned up code for SGD Based Answer extender

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue Generation CI

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* fix Jenkins CI issue"

Signed-off-by: Zhilin Wang <[email protected]>

* add support for design dataset

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary imports

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support megatron for dialogue_s2s_generation_model

Signed-off-by: Zhilin Wang <[email protected]>

* reduce loaded samples in MSMarcoDataProcessor to 64 when cfg.model.dataset.debug_mode=True

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update CI

Signed-off-by: Zhilin Wang <[email protected]>

* update checkpoint and predictions filename to include epoch number

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate HF BART MNLI into zero shot intent model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Nearest Neighbour Model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* refactor Dialogue SGD Data Processor to make interface for models cleaner

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue S2S Generation model for DialogueSGDDataProcessor interface

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support sgd and drive thru datasets by zero shot model and nearest neighbour model

Signed-off-by: Zhilin Wang <[email protected]>

* add prediction saving code to nearest neighbour and zero shot intent models

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo in sgd data processor

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Mellon QA Data Processor

Signed-off-by: Zhilin Wang <[email protected]>

* update mellon qa

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue.py to remove outdated info

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* add dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* address review comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix for cfg

Signed-off-by: Zhilin Wang <[email protected]>

* make dependency on apex optional

Signed-off-by: Zhilin Wang <[email protected]>

* change NLPDDPluggin calling logic to make it possible to run without apex

Signed-off-by: Zhilin Wang <[email protected]>

* add first draft of tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* reduce ms marco size by removing lines without wellFormedAnswers

Signed-off-by: Zhilin Wang <[email protected]>

* address pr comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update colab tutorial link in dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* include unit test and some refactor to facilitate unit test

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* address pr issues

Signed-off-by: Zhilin Wang <[email protected]>

* remove typos in dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* support larger files for question answering

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary artifacts to reduce memory use

Signed-off-by: Zhilin Wang <[email protected]>

* put 0 tensor to device

Signed-off-by: Zhilin Wang <[email protected]>

* update link within dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously delete files

Signed-off-by: Zhilin Wang <[email protected]>

* update error handling when loss = nan

Signed-off-by: Zhilin Wang <[email protected]>

* update nan handling

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss func

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss

Signed-off-by: Zhilin Wang <[email protected]>

* fix type error raised in qa_dataset.py

Signed-off-by: Zhilin Wang <[email protected]>

* add error checking message

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update exp logging

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* limit number of negative samples

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused methods and style fix

Signed-off-by: Zhilin Wang <[email protected]>

* add more documentation

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* changes base on PR review

Signed-off-by: Zhilin Wang <[email protected]>

* set wandb logger falseby default

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

* style fix

* correct typo

* style fix

* style fix

Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>

* Fix ASR Typos in tutorials (#4384)

* Fix typos

Signed-off-by: smajumdar <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
(cherry picked from commit 0322b158f26a0b690edca7a84714e33752283923)

Co-authored-by: Travis Bartley <[email protected]>

* Add Docs for NeMo Adapters (#4369)

Signed-off-by: smajumdar <[email protected]>

* Update NeMo docs (#4397)

Signed-off-by: smajumdar <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* update branch

Signed-off-by: ericharper <[email protected]>

* remove Copy of

Signed-off-by: ericharper <[email protected]>

Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Micha Livne <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Travis Bartley <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* [bugfix][TTS] pitch, voiced_mask, prob_voiced have the same values. (#4392)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Fixing import error in some cases (#4401)

Signed-off-by: Boris Fomitchev <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Fixing bugs in calling method ctc_decoder_predictions_tensor. (#4414)

* updated ctc decoding calls.

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Update with new conformer checkpoints. (#4417)

Signed-off-by: Matvei Novikov <[email protected]>

* [TTS] add static method decorator. (#4443)

* [TTS] add static method decorator.

Signed-off-by: Xuesong Yang <[email protected]>

* remove protect prefix

Signed-off-by: Xuesong Yang <[email protected]>

* fixed style error

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

Co-authored-by: Taejin Park <[email protected]>
Co-authored-by: Nithin Rao <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Georg Kucsko <[email protected]>
Co-authored-by: fayejf <[email protected]>
Co-authored-by: Xiaowei Ren <[email protected]>
Co-authored-by: Shantanu Acharya <[email protected]>
Co-authored-by: Travis Bartley <[email protected]>
Co-authored-by: tbartley94 <[email protected]>
Co-authored-by: Xuesong Yang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Micha Livne <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Boris Fomitchev <[email protected]>
Co-authored-by: Vahid Noroozi <[email protected]>

* Add ITN pt (#4516)

* Add ITN pt

Signed-off-by: Guilherme Steinmann <[email protected]>

* Fix style

Signed-off-by: Guilherme Steinmann <[email protected]>

* Fix style

Signed-off-by: Guilherme Steinmann <[email protected]>

* Update copyright year to 2022 on ITN pt rules and tests

Signed-off-by: Guilherme Steinmann <[email protected]>

* Fixed WER initialization in ASR_with_Nemo notebook (#4523)

Signed-off-by: Ante Jukić <[email protected]>

Co-authored-by: Ante Jukić <[email protected]>

* Update cmudict (#4510)

phoneme IY1 -> IH1 in NVIDIA
Added phonemes for CUSTOMIZABLE

Update cmudict file revision and its reference.

Signed-off-by: Jason Roche <[email protected]>

Co-authored-by: Jason Roche <[email protected]>
Co-authored-by: Xuesong Yang <[email protected]>

* [Add] Support for Different LRs with Param Groups (#4508)

* add support for param groups

Signed-off-by: stevehuang52 <[email protected]>

* make config more general

Signed-off-by: stevehuang52 <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* Weighted bucketing (#4474)

* Add silence handling for speaker diarization pipeline (#4512)

* initial commit

Signed-off-by: nithinraok <[email protected]>

* fixed silence wav file issue causing clustering to evaluate on null embeddings

Signed-off-by: nithinraok <[email protected]>

* fixed zero duration issue

Signed-off-by: nithinraok <[email protected]>

* updated with comments

Signed-off-by: nithinraok <[email protected]>

* minor doc change

Signed-off-by: nithinraok <[email protected]>

* update log

Signed-off-by: nithinraok <[email protected]>

* Fix runtime check (#4501)

* Runtime check refinements

Signed-off-by: Boris Fomitchev <[email protected]>

* Added fp32 casting for ASR nets export

Signed-off-by: Boris Fomitchev <[email protected]>

* style

Signed-off-by: Boris Fomitchev <[email protected]>

* Used torch.float32 for clarity

Signed-off-by: Boris Fomitchev <[email protected]>

* Fixing parameters passing

Signed-off-by: Boris Fomitchev <[email protected]>

* Update finetune label models (#4504)

* initial_script

Signed-off-by: nithinraok <[email protected]>

* move old script

Signed-off-by: nithinraok <[email protected]>

* remove finetune func from label models

Signed-off-by: nithinraok <[email protected]>

* style clean

Signed-off-by: nithinraok <[email protected]>

* updated config

Signed-off-by: nithinraok <[email protected]>

* update tutorial

Signed-off-by: nithinraok <[email protected]>

* lgtm fixes

Signed-off-by: nithinraok <[email protected]>

* updated based on comments

Signed-off-by: nithinraok <[email protected]>

* update doc

Signed-off-by: nithinraok <[email protected]>

* [ASR][Breaking Change] Update signature of Hypothesis alignments (#4511)

* Preserve logprobs when preserving alignments

Signed-off-by: smajumdar <[email protected]>

* Update tests for rnnt gredy and beam search

Signed-off-by: smajumdar <[email protected]>

* Update all dependents of alignments

Signed-off-by: smajumdar <[email protected]>

* Update docs

Signed-off-by: smajumdar <[email protected]>

* Weighted bucketing (#4530)

* Additional sentencepiece args - Byte fallback, split digits, split_on_whitespace (#4525)

* Fix geglu without fusion

Signed-off-by: MaximumEntropy <[email protected]>

* Add extra args

Signed-off-by: MaximumEntropy <[email protected]>

* Reset transformer

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Fix spm arg

Signed-off-by: MaximumEntropy <[email protected]>

* Fix help string

Signed-off-by: MaximumEntropy <[email protected]>

* Add support for ASR Adapter Auxiliary Losses (#4480)

* Add support for access mixin registry of custom losses

Signed-off-by: smajumdar <[email protected]>

* add support for asr custom losses

Signed-off-by: smajumdar <[email protected]>

* Update for l2 loss

Signed-off-by: smajumdar <[email protected]>

* Add unittests

Signed-off-by: smajumdar <[email protected]>

* Add unittests

Signed-off-by: smajumdar <[email protected]>

* Add unittests

Signed-off-by: smajumdar <[email protected]>

* Update registration of tensors to reset after finishing step

Signed-off-by: smajumdar <[email protected]>

* Remove comment

Signed-off-by: smajumdar <[email protected]>

* Remove comment

Signed-off-by: smajumdar <[email protected]>

* Update SSL models

Signed-off-by: smajumdar <[email protected]>

* Add support for validation step properly registering tensors

Signed-off-by: smajumdar <[email protected]>

* Move reset of registry outside

Signed-off-by: smajumdar <[email protected]>

* update (#4520)

Signed-off-by: stevehuang52 <[email protected]>

* fix duplex inference with grammars (#4517)

* fix duplex inference with grammars

Signed-off-by: ekmb <[email protected]>

* add ci test for duplex, fix electronic last sym bug

Signed-off-by: ekmb <[email protected]>

* test fix

Signed-off-by: ekmb <[email protected]>

* fix jenkins

Signed-off-by: ekmb <[email protected]>

* update jenkins grammars

Signed-off-by: ekmb <[email protected]>

* add pt to the docs

Signed-off-by: ekmb <[email protected]>

* fix jenkins

Signed-off-by: ekmb <[email protected]>

* disable test

Signed-off-by: ekmb <[email protected]>

* fix jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins refactor

Signed-off-by: ekmb <[email protected]>

* fix jenkins

Signed-off-by: ekmb <[email protected]>

* fix jenkins

Signed-off-by: ekmb <[email protected]>

* fix jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins

Signed-off-by: ekmb <[email protected]>

* test

Signed-off-by: ekmb <[email protected]>

* test

Signed-off-by: ekmb <[email protected]>

* test

Signed-off-by: ekmb <[email protected]>

* test

Signed-off-by: ekmb <[email protected]>

Co-authored-by: Yang Zhang <[email protected]>

* Add Bucketing support to TarredAudioToClassificationLabelDataset (#4465)

* Add Bucketing support to TarredAudioToClassificationLabelDataset

Signed-off-by: Ewald Enzinger <[email protected]>

* Add MTEncDec Finetune support (#4540)

* add FT support

Signed-off-by: Abhinav Khattar <[email protected]>

* rm preproc

Signed-off-by: Abhinav Khattar <[email protected]>

* review changes

Signed-off-by: Abhinav Khattar <[email protected]>

* add CI

Signed-off-by: Abhinav Khattar <[email protected]>

* newline fix

Signed-off-by: Abhinav Khattar <[email protected]>

* CI fix

Signed-off-by: Abhinav Khattar <[email protected]>

* clean up

Signed-off-by: Abhinav Khattar <[email protected]>

* post training cleanup

Signed-off-by: Abhinav Khattar <[email protected]>

* test

Signed-off-by: Abhinav Khattar <[email protected]>

* revert

Signed-off-by: Abhinav Khattar <[email protected]>

* CI test

Signed-off-by: Abhinav Khattar <[email protected]>

* revert CI changes

Signed-off-by: Abhinav Khattar <[email protected]>

* original CI

Signed-off-by: Abhinav Khattar <[email protected]>

Co-authored-by: Sandeep Subramanian <[email protected]>

* Add nsys profiling (#4539)

* add nsys profiling

Signed-off-by: ericharper <[email protected]>

* only access omegaconf in setup

Signed-off-by: ericharper <[email protected]>

* use robust get_rank function

Signed-off-by: ericharper <[email protected]>

* simplify

Signed-off-by: ericharper <[email protected]>

* Update megatron prompt learning interface to dialogue  (#4545)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assistant data processsor

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins for SGDGEN local checkpoint

Signed-off-by: Zhilin Wang …
paarthneekhara added a commit to paarthneekhara/NeMo that referenced this pull request Jul 28, 2022
* Megatron BART BOS / EOS bug fix (#4495)

* 1. Debugging.

Signed-off-by: Micha Livne <[email protected]>

* 1. BART dataset fixes missing <EOS> for deocder output.

Signed-off-by: Micha Livne <[email protected]>

* 1. Debugging.

Signed-off-by: Micha Livne <[email protected]>

* 1. Debugging.

Signed-off-by: Micha Livne <[email protected]>

* 1. Removed extra padding from BARTDataset.

Signed-off-by: Micha Livne <[email protected]>

* GPT Prompt Learning Improvements (#4496)

* Updated pipeline parallel code to speed up training

Signed-off-by: Virginia Adams <[email protected]>

* Load global batch size not local mini batch size

Signed-off-by: Virginia Adams <[email protected]>

* Python reformatting

Signed-off-by: Virginia Adams <[email protected]>

* Megatron perceiver with tensor parallelism only (#4318)

* Temp

Signed-off-by: MaximumEntropy <[email protected]>

* Add megatron dataset

Signed-off-by: MaximumEntropy <[email protected]>

* Update config and fix global batch fetcher

Signed-off-by: MaximumEntropy <[email protected]>

* Add dataset class

Signed-off-by: MaximumEntropy <[email protected]>

* Update comments

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Update yaml

Signed-off-by: MaximumEntropy <[email protected]>

* Fix duplicate yaml key

Signed-off-by: MaximumEntropy <[email protected]>

* Translate method and preprocess script for raw text

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Remove pdb

Signed-off-by: MaximumEntropy <[email protected]>

* Fix arg name

Signed-off-by: MaximumEntropy <[email protected]>

* Fix other arg

Signed-off-by: MaximumEntropy <[email protected]>

* Change sampler back

Signed-off-by: MaximumEntropy <[email protected]>

* Move back to global batch fetcher to use distributed sampler

Signed-off-by: MaximumEntropy <[email protected]>

* Add text memmap data

Signed-off-by: MaximumEntropy <[email protected]>

* Update monitor

Signed-off-by: MaximumEntropy <[email protected]>

* Fixes for PP

Signed-off-by: MaximumEntropy <[email protected]>

* Remove unused import

Signed-off-by: MaximumEntropy <[email protected]>

* Truncate examples in text memmap

Signed-off-by: MaximumEntropy <[email protected]>

* NMT training batch interpolation key

Signed-off-by: MaximumEntropy <[email protected]>

* tarred data fix

Signed-off-by: MaximumEntropy <[email protected]>

* Change dataset type check

Signed-off-by: MaximumEntropy <[email protected]>

* Fix sampler

Signed-off-by: MaximumEntropy <[email protected]>

* Pass dataset cfg to determine type

Signed-off-by: MaximumEntropy <[email protected]>

* Log global step on validation step as well

Signed-off-by: MaximumEntropy <[email protected]>

* Fix NMT model saving with artifacts

Signed-off-by: MaximumEntropy <[email protected]>

* Initialize DDP in decode if not initialized. Needed for inference only mode

Signed-off-by: MaximumEntropy <[email protected]>

* Megatron NMT inference script

Signed-off-by: MaximumEntropy <[email protected]>

* Inference config file

Signed-off-by: MaximumEntropy <[email protected]>

* hardcode max delta temporarily

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* detokenizer if processor is not none

Signed-off-by: MaximumEntropy <[email protected]>

* Sampler config

Signed-off-by: MaximumEntropy <[email protected]>

* Compat with configs without sampler arg

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Comment for validation dataset type

Signed-off-by: MaximumEntropy <[email protected]>

* Fix tokenizer building

Signed-off-by: MaximumEntropy <[email protected]>

* CI test for megatron nmt

Signed-off-by: MaximumEntropy <[email protected]>

* Fix tokenizer in restore

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* O2 restore from fix

Signed-off-by: MaximumEntropy <[email protected]>

* Remove print

Signed-off-by: MaximumEntropy <[email protected]>

* Change tokenizer model name in config

Signed-off-by: MaximumEntropy <[email protected]>

* Logging

Signed-off-by: MaximumEntropy <[email protected]>

* Set seed for distributed sampler

Signed-off-by: MaximumEntropy <[email protected]>

* Cluster debugging messages

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Fix max generation delta

Signed-off-by: MaximumEntropy <[email protected]>

* No LM Init

Signed-off-by: MaximumEntropy <[email protected]>

* Use nlp save restore connector

Signed-off-by: MaximumEntropy <[email protected]>

* Remove useless infer args

Signed-off-by: MaximumEntropy <[email protected]>

* Typo

Signed-off-by: MaximumEntropy <[email protected]>

* UTF8 safe print of translation result

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Add save restore connector back with comment

Signed-off-by: MaximumEntropy <[email protected]>

* Refactor

Signed-off-by: MaximumEntropy <[email protected]>

* Fix CI test

Signed-off-by: MaximumEntropy <[email protected]>

* Add missing args

Signed-off-by: MaximumEntropy <[email protected]>

* Address comments

Signed-off-by: MaximumEntropy <[email protected]>

* Empty to restart

* Fix CI test

Signed-off-by: MaximumEntropy <[email protected]>

* Check for test ds

Signed-off-by: MaximumEntropy <[email protected]>

* set fusion to false

Signed-off-by: MaximumEntropy <[email protected]>

* Initial perceiver encoder

Signed-off-by: MaximumEntropy <[email protected]>

* Perceiver with PP=1

Signed-off-by: MaximumEntropy <[email protected]>

* Remove init cross attn

Signed-off-by: MaximumEntropy <[email protected]>

* CI test and remove init cross attn arg

Signed-off-by: MaximumEntropy <[email protected]>

* Remove init cross attn layers from file

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Clean up

Signed-off-by: MaximumEntropy <[email protected]>

* update branch

Signed-off-by: ericharper <[email protected]>

* Set headscale false (#4364)

Signed-off-by: MaximumEntropy <[email protected]>

* Add wandb as dependency (#4365)

Signed-off-by: smajumdar <[email protected]>

* Raise trainer error (#4356)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Set headscale false (#4364) (#4366)

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: smajumdar <[email protected]>

* Finetuning changes for BART (#4003)

* Temp

Signed-off-by: MaximumEntropy <[email protected]>

* Checkpoint converter to nemo for bart

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Make position embedding expansion specific to a batch to avoid checkpoint size mismatches (#4357)

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Fix logging warning

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Refactor bias act fusion

Signed-off-by: MaximumEntropy <[email protected]>

* Update NMT config

Signed-off-by: MaximumEntropy <[email protected]>

* Fix electronic bug, new time ITN rule (#4355)

* fix electronic bug

Signed-off-by: ekmb <[email protected]>

* add new itn time rule

Signed-off-by: ekmb <[email protected]>

* revert domain changes

Signed-off-by: ekmb <[email protected]>

* remove repetition

Signed-off-by: ekmb <[email protected]>

* Update ci tests

Signed-off-by: MaximumEntropy <[email protected]>

* Correct support for dataclasses in default module dim (#4372)

* Correct support for dataclasses in default module dim

Signed-off-by: smajumdar <[email protected]>

* Fix path for save of results

Signed-off-by: smajumdar <[email protected]>

* fix pad id bug (#4377)

Signed-off-by: Yi Dong <[email protected]>

* Question answering bug fix (#4381)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assistant data processsor

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins for SGDGEN local checkpoint

Signed-off-by: Zhilin Wang <[email protected]>

* update style

Signed-off-by: Zhilin Wang <[email protected]>

* use local vocab file for Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* patch for Jenkins CI using local file

Signed-off-by: Zhilin Wang <[email protected]>

* add slot filling prediction and metrics

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused code

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* refactor metrics code out of Dialogue GPT Model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate backward compatible support for IntentSlotClassificationModel (bert model)

Signed-off-by: Zhilin Wang <[email protected]>

* save prediction file for IntentSlotClassification

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue gpt model training for megatron gpt

Signed-off-by: Zhilin Wang <[email protected]>

* remove batch generate for HF GPT2, which causes lower performance

Signed-off-by: Zhilin Wang <[email protected]>

* add few shot capability to dialogue gpt model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile and remove unused import

Signed-off-by: Zhilin Wang <[email protected]>

* update code description and clarity

Signed-off-by: Zhilin Wang <[email protected]>

* address PR comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate compatibility with ZeroShotIntentModel

Signed-off-by: Zhilin Wang <[email protected]>

* rename folder to dialogue due to increased scope and further refactor for clarity

Signed-off-by: Zhilin Wang <[email protected]>

* added dialogue GPT for sequence generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* add CI test for DialogueGPTGenerationModel

Signed-off-by: Zhilin Wang <[email protected]>

* integrate DialogueS2SGenerationModel for generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* modify huggingface utils to support HF t5/BART models

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* fix bleu metric style

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* update based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 2 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 3 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* integrate sgd generation based on user user utterance and system slot-values to generate system utterance

Signed-off-by: Zhilin Wang <[email protected]>

* add validation model saving capabilities

Signed-off-by: Zhilin Wang <[email protected]>

* cleaned up code for SGD Based Answer extender

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue Generation CI

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* fix Jenkins CI issue"

Signed-off-by: Zhilin Wang <[email protected]>

* add support for design dataset

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary imports

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support megatron for dialogue_s2s_generation_model

Signed-off-by: Zhilin Wang <[email protected]>

* reduce loaded samples in MSMarcoDataProcessor to 64 when cfg.model.dataset.debug_mode=True

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update CI

Signed-off-by: Zhilin Wang <[email protected]>

* update checkpoint and predictions filename to include epoch number

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate HF BART MNLI into zero shot intent model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Nearest Neighbour Model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* refactor Dialogue SGD Data Processor to make interface for models cleaner

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue S2S Generation model for DialogueSGDDataProcessor interface

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support sgd and drive thru datasets by zero shot model and nearest neighbour model

Signed-off-by: Zhilin Wang <[email protected]>

* add prediction saving code to nearest neighbour and zero shot intent models

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo in sgd data processor

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Mellon QA Data Processor

Signed-off-by: Zhilin Wang <[email protected]>

* update mellon qa

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue.py to remove outdated info

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* add dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* address review comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix for cfg

Signed-off-by: Zhilin Wang <[email protected]>

* make dependency on apex optional

Signed-off-by: Zhilin Wang <[email protected]>

* change NLPDDPluggin calling logic to make it possible to run without apex

Signed-off-by: Zhilin Wang <[email protected]>

* add first draft of tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* reduce ms marco size by removing lines without wellFormedAnswers

Signed-off-by: Zhilin Wang <[email protected]>

* address pr comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update colab tutorial link in dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* include unit test and some refactor to facilitate unit test

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* address pr issues

Signed-off-by: Zhilin Wang <[email protected]>

* remove typos in dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* support larger files for question answering

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary artifacts to reduce memory use

Signed-off-by: Zhilin Wang <[email protected]>

* put 0 tensor to device

Signed-off-by: Zhilin Wang <[email protected]>

* update link within dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously delete files

Signed-off-by: Zhilin Wang <[email protected]>

* update error handling when loss = nan

Signed-off-by: Zhilin Wang <[email protected]>

* update nan handling

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss func

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss

Signed-off-by: Zhilin Wang <[email protected]>

* fix type error raised in qa_dataset.py

Signed-off-by: Zhilin Wang <[email protected]>

* add error checking message

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update exp logging

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* limit number of negative samples

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused methods and style fix

Signed-off-by: Zhilin Wang <[email protected]>

* add more documentation

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* changes base on PR review

Signed-off-by: Zhilin Wang <[email protected]>

* set wandb logger falseby default

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

* style fix

* correct typo

* style fix

* style fix

Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>

* Fix ASR Typos in tutorials (#4384)

* Fix typos

Signed-off-by: smajumdar <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
(cherry picked from commit 0322b158f26a0b690edca7a84714e33752283923)

Co-authored-by: Travis Bartley <[email protected]>

* Add Docs for NeMo Adapters (#4369)

Signed-off-by: smajumdar <[email protected]>

* Update NeMo docs (#4397)

Signed-off-by: smajumdar <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* Punctuation and capitalization tests race condition (#4399)

* Add draft of race condition fixes

Signed-off-by: PeganovAnton <[email protected]>

* Minor improvements

Signed-off-by: PeganovAnton <[email protected]>

* More race condition fixes

Signed-off-by: PeganovAnton <[email protected]>

* Improve error message

Signed-off-by: PeganovAnton <[email protected]>

* Improve error message

Signed-off-by: PeganovAnton <[email protected]>

* Improve error message

Signed-off-by: PeganovAnton <[email protected]>

* bias act fusion changes

Signed-off-by: MaximumEntropy <[email protected]>

* Address comments

Signed-off-by: MaximumEntropy <[email protected]>

* Fix geglu without fusion

Signed-off-by: MaximumEntropy <[email protected]>

* Reset files to main

Signed-off-by: MaximumEntropy <[email protected]>

* Remove hidden blocks

Signed-off-by: MaximumEntropy <[email protected]>

* Fix style

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>
Co-authored-by: Abhinav Khattar <[email protected]>
Co-authored-by: ericharper <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Travis Bartley <[email protected]>
Co-authored-by: PeganovAnton <[email protected]>

* NMESC speaker counting algorithm update (#4500)

* initial commit

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Default maj_vote = False, max_rp=0.25

Signed-off-by: Taejin Park <[email protected]>

* doc strings and style fix

Signed-off-by: Taejin Park <[email protected]>

* Docstring minor edit

Signed-off-by: Taejin Park <[email protected]>

* Default False in the functions

Signed-off-by: Taejin Park <[email protected]>

* fixed repeated variable

Signed-off-by: Taejin Park <[email protected]>

* Default as maj_vote=False

Signed-off-by: Taejin Park <[email protected]>

* removed redundant part in wrtie_rttm func

Signed-off-by: Taejin Park <[email protected]>

* Removed unused function

Signed-off-by: Taejin Park <[email protected]>

* Updated and tested silence and very short samples

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Style fix and removing unnecessary parts

Signed-off-by: Taejin Park <[email protected]>

* unused variables are removed

Signed-off-by: Taejin Park <[email protected]>

* Fixed commented torch.jit.script

Signed-off-by: Taejin Park <[email protected]>

* majority voting update

Signed-off-by: Taejin Park <[email protected]>

* cancelling the update on speaker_utils and clus_diarizer

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* bug fix

Signed-off-by: Taejin Park <[email protected]>

* Added fp32 converting for torch.mm

Signed-off-by: Taejin Park <[email protected]>

Co-authored-by: Nithin Rao <[email protected]>

* Fix dataset parameter typo on tacotron2 example yaml (#4471)

Signed-off-by: saarus72 <[email protected]>

Co-authored-by: Xuesong Yang <[email protected]>

* Noam lr sched: do not force min_lr after max_steps (#4472)

Signed-off-by: Adrian Lancucki <[email protected]>

Co-authored-by: Adrian Lancucki <[email protected]>
Co-authored-by: Xuesong Yang <[email protected]>

* Refactor for punctuation model (#4367)

* Dataloader, collector, loss and metric for multiscale diarization decoder  (#4187)

* First commit

Signed-off-by: Taejin Park <[email protected]>

* Checked funtionality and imports

Signed-off-by: Taejin Park <[email protected]>

* fixed import issues

Signed-off-by: Taejin Park <[email protected]>

* Removed the changed made by mistake

Signed-off-by: Taejin Park <[email protected]>

* Style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM errors 001

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM and style fix

Signed-off-by: Taejin Park <[email protected]>

* Changed docstrings

Signed-off-by: Taejin Park <[email protected]>

* LGTM again

Signed-off-by: Taejin Park <[email protected]>

* Removed unnecessary torch setting lines

Signed-off-by: Taejin Park <[email protected]>

* Style fix and isort

Signed-off-by: Taejin Park <[email protected]>

* jbalam-nv comments reflected

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected comments and created _diar_label.py

Signed-off-by: Taejin Park <[email protected]>

* Typo fix and style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed target_spks[0] index error

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* LGTM unused import IterDataset

Signed-off-by: Taejin Park <[email protected]>

* revert collection doc year

Signed-off-by: Taejin Park <[email protected]>

* Code format error in collections.py

Signed-off-by: Taejin Park <[email protected]>

* fix collections space format error

Signed-off-by: Taejin Park <[email protected]>

* merged main correctly

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected all comments and tested

Signed-off-by: Taejin Park <[email protected]>

* style fix and LGTM

Signed-off-by: Taejin Park <[email protected]>

* rttm_filepath to rttm_file and removed self included funcs, tested

Signed-off-by: Taejin Park <[email protected]>

Co-authored-by: Nithin Rao <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* removed references to data_dir

Signed-off-by: Matvei Novikov <[email protected]>

* added missing parameters to data preparation script

Signed-off-by: Matvei Novikov <[email protected]>

* removed unnecessary file extension check

Signed-off-by: Matvei Novikov <[email protected]>

* Add ASR CTC Decoding module (#4342)

* Initial commit

Signed-off-by: smajumdar <[email protected]>

* Full support for decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Temp

Signed-off-by: smajumdar <[email protected]>

* Fix labels of y_sequence

Signed-off-by: smajumdar <[email protected]>

* Set support for sentencepiece subword merging

Signed-off-by: smajumdar <[email protected]>

* Fix char and word based token merge alignment

Signed-off-by: smajumdar <[email protected]>

* Revert incorrect change

Signed-off-by: smajumdar <[email protected]>

* Update docstring

Signed-off-by: smajumdar <[email protected]>

* Improve compatibility with greedy tokens and log probs

Signed-off-by: smajumdar <[email protected]>

* Update scripts to use decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix decoding of ctc models

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Option to disable mp in VAD via num_workers=1 (#4317)

* Option to disable mp in VAD via num_workers=1

In certain environments python multiprocessing can deadlock. This adds a convenient version to disable by setting num_workers to 1.

Signed-off-by: Georg Kucsko <[email protected]>

* add none handling

Signed-off-by: Georg Kucsko <[email protected]>

* additional none handling

Signed-off-by: Georg Kucsko <[email protected]>

Co-authored-by: fayejf <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* remove redundant bias expand (#4382)

* remove redundant bias expand

Signed-off-by: Xiaowei Ren <[email protected]>

* delete redundant code

Signed-off-by: Xiaowei Ren <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* fixed style

Signed-off-by: Matvei Novikov <[email protected]>

* Add option for specifying wandb save_dir from config (#4379)

* give option to user to specify wandb save dir via config

Signed-off-by: Shantanu Acharya <[email protected]>

* create save_dir directory for wandb logger if not exists

Signed-off-by: Shantanu Acharya <[email protected]>

* update save_dir get method with a default value

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* [Bugfix][TTS] wrong order of returned tuple for general_collate_fn. (#4388)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Merge r1.10.0 main (#4398)

* update branch

Signed-off-by: ericharper <[email protected]>

* Set headscale false (#4364)

Signed-off-by: MaximumEntropy <[email protected]>

* Add wandb as dependency (#4365)

Signed-off-by: smajumdar <[email protected]>

* Raise trainer error (#4356)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Set headscale false (#4364) (#4366)

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: smajumdar <[email protected]>

* Finetuning changes for BART (#4003)

* Temp

Signed-off-by: MaximumEntropy <[email protected]>

* Checkpoint converter to nemo for bart

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Make position embedding expansion specific to a batch to avoid checkpoint size mismatches (#4357)

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Fix logging warning

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Fix electronic bug, new time ITN rule (#4355)

* fix electronic bug

Signed-off-by: ekmb <[email protected]>

* add new itn time rule

Signed-off-by: ekmb <[email protected]>

* revert domain changes

Signed-off-by: ekmb <[email protected]>

* remove repetition

Signed-off-by: ekmb <[email protected]>

* Correct support for dataclasses in default module dim (#4372)

* Correct support for dataclasses in default module dim

Signed-off-by: smajumdar <[email protected]>

* Fix path for save of results

Signed-off-by: smajumdar <[email protected]>

* fix pad id bug (#4377)

Signed-off-by: Yi Dong <[email protected]>

* Question answering bug fix (#4381)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assistant data processsor

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins for SGDGEN local checkpoint

Signed-off-by: Zhilin Wang <[email protected]>

* update style

Signed-off-by: Zhilin Wang <[email protected]>

* use local vocab file for Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* patch for Jenkins CI using local file

Signed-off-by: Zhilin Wang <[email protected]>

* add slot filling prediction and metrics

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused code

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* refactor metrics code out of Dialogue GPT Model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate backward compatible support for IntentSlotClassificationModel (bert model)

Signed-off-by: Zhilin Wang <[email protected]>

* save prediction file for IntentSlotClassification

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue gpt model training for megatron gpt

Signed-off-by: Zhilin Wang <[email protected]>

* remove batch generate for HF GPT2, which causes lower performance

Signed-off-by: Zhilin Wang <[email protected]>

* add few shot capability to dialogue gpt model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile and remove unused import

Signed-off-by: Zhilin Wang <[email protected]>

* update code description and clarity

Signed-off-by: Zhilin Wang <[email protected]>

* address PR comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate compatibility with ZeroShotIntentModel

Signed-off-by: Zhilin Wang <[email protected]>

* rename folder to dialogue due to increased scope and further refactor for clarity

Signed-off-by: Zhilin Wang <[email protected]>

* added dialogue GPT for sequence generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* add CI test for DialogueGPTGenerationModel

Signed-off-by: Zhilin Wang <[email protected]>

* integrate DialogueS2SGenerationModel for generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* modify huggingface utils to support HF t5/BART models

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* fix bleu metric style

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* update based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 2 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 3 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* integrate sgd generation based on user user utterance and system slot-values to generate system utterance

Signed-off-by: Zhilin Wang <[email protected]>

* add validation model saving capabilities

Signed-off-by: Zhilin Wang <[email protected]>

* cleaned up code for SGD Based Answer extender

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue Generation CI

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* fix Jenkins CI issue"

Signed-off-by: Zhilin Wang <[email protected]>

* add support for design dataset

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary imports

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support megatron for dialogue_s2s_generation_model

Signed-off-by: Zhilin Wang <[email protected]>

* reduce loaded samples in MSMarcoDataProcessor to 64 when cfg.model.dataset.debug_mode=True

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update CI

Signed-off-by: Zhilin Wang <[email protected]>

* update checkpoint and predictions filename to include epoch number

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate HF BART MNLI into zero shot intent model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Nearest Neighbour Model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* refactor Dialogue SGD Data Processor to make interface for models cleaner

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue S2S Generation model for DialogueSGDDataProcessor interface

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support sgd and drive thru datasets by zero shot model and nearest neighbour model

Signed-off-by: Zhilin Wang <[email protected]>

* add prediction saving code to nearest neighbour and zero shot intent models

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo in sgd data processor

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Mellon QA Data Processor

Signed-off-by: Zhilin Wang <[email protected]>

* update mellon qa

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue.py to remove outdated info

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* add dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* address review comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix for cfg

Signed-off-by: Zhilin Wang <[email protected]>

* make dependency on apex optional

Signed-off-by: Zhilin Wang <[email protected]>

* change NLPDDPluggin calling logic to make it possible to run without apex

Signed-off-by: Zhilin Wang <[email protected]>

* add first draft of tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* reduce ms marco size by removing lines without wellFormedAnswers

Signed-off-by: Zhilin Wang <[email protected]>

* address pr comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update colab tutorial link in dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* include unit test and some refactor to facilitate unit test

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* address pr issues

Signed-off-by: Zhilin Wang <[email protected]>

* remove typos in dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* support larger files for question answering

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary artifacts to reduce memory use

Signed-off-by: Zhilin Wang <[email protected]>

* put 0 tensor to device

Signed-off-by: Zhilin Wang <[email protected]>

* update link within dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously delete files

Signed-off-by: Zhilin Wang <[email protected]>

* update error handling when loss = nan

Signed-off-by: Zhilin Wang <[email protected]>

* update nan handling

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss func

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss

Signed-off-by: Zhilin Wang <[email protected]>

* fix type error raised in qa_dataset.py

Signed-off-by: Zhilin Wang <[email protected]>

* add error checking message

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update exp logging

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* limit number of negative samples

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused methods and style fix

Signed-off-by: Zhilin Wang <[email protected]>

* add more documentation

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* changes base on PR review

Signed-off-by: Zhilin Wang <[email protected]>

* set wandb logger falseby default

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

* style fix

* correct typo

* style fix

* style fix

Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>

* Fix ASR Typos in tutorials (#4384)

* Fix typos

Signed-off-by: smajumdar <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
(cherry picked from commit 0322b158f26a0b690edca7a84714e33752283923)

Co-authored-by: Travis Bartley <[email protected]>

* Add Docs for NeMo Adapters (#4369)

Signed-off-by: smajumdar <[email protected]>

* Update NeMo docs (#4397)

Signed-off-by: smajumdar <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* update branch

Signed-off-by: ericharper <[email protected]>

* remove Copy of

Signed-off-by: ericharper <[email protected]>

Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Micha Livne <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Travis Bartley <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* [bugfix][TTS] pitch, voiced_mask, prob_voiced have the same values. (#4392)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Fixing import error in some cases (#4401)

Signed-off-by: Boris Fomitchev <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Fixing bugs in calling method ctc_decoder_predictions_tensor. (#4414)

* updated ctc decoding calls.

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Update with new conformer checkpoints. (#4417)

Signed-off-by: Matvei Novikov <[email protected]>

* [TTS] add static method decorator. (#4443)

* [TTS] add static method decorator.

Signed-off-by: Xuesong Yang <[email protected]>

* remove protect prefix

Signed-off-by: Xuesong Yang <[email protected]>

* fixed style error

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

Co-authored-by: Taejin Park <[email protected]>
Co-authored-by: Nithin Rao <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Georg Kucsko <[email protected]>
Co-authored-by: fayejf <[email protected]>
Co-authored-by: Xiaowei Ren <[email protected]>
Co-authored-by: Shantanu Acharya <[email protected]>
Co-authored-by: Travis Bartley <[email protected]>
Co-authored-by: tbartley94 <[email protected]>
Co-authored-by: Xuesong Yang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Micha Livne <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Boris Fomitchev <[email protected]>
Co-authored-by: Vahid Noroozi <[email protected]>

* bug fix - sample rate was being ignored in vocoder dataset when not loading mel

Signed-off-by: Paarth Neekhara <[email protected]>

* Add ITN pt (#4516)

* Add ITN pt

Signed-off-by: Guilherme Steinmann <[email protected]>

* Fix style

Signed-off-by: Guilherme Steinmann <[email protected]>

* Fix style

Signed-off-by: Guilherme Steinmann <[email protected]>

* Update copyright year to 2022 on ITN pt rules and tests

Signed-off-by: Guilherme Steinmann <[email protected]>

* Fixed WER initialization in ASR_with_Nemo notebook (#4523)

Signed-off-by: Ante Jukić <[email protected]>

Co-authored-by: Ante Jukić <[email protected]>

* Update cmudict (#4510)

phoneme IY1 -> IH1 in NVIDIA
Added phonemes for CUSTOMIZABLE

Update cmudict file revision and its reference.

Signed-off-by: Jason Roche <[email protected]>

Co-authored-by: Jason Roche <[email protected]>
Co-authored-by: Xuesong Yang <[email protected]>

* [Add] Support for Different LRs with Param Groups (#4508)

* add support for param groups

Signed-off-by: stevehuang52 <[email protected]>

* make config more general

Signed-off-by: stevehuang52 <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* Weighted bucketing (#4474)

* Add silence handling for speaker diarization pipeline (#4512)

* initial commit

Signed-off-by: nithinraok <[email protected]>

* fixed silence wav file issue causing clustering to evaluate on null embeddings

Signed-off-by: nithinraok <[email protected]>

* fixed zero duration issue

Signed-off-by: nithinraok <[email protected]>

* updated with comments

Signed-off-by: nithinraok <[email protected]>

* minor doc change

Signed-off-by: nithinraok <[email protected]>

* update log

Signed-off-by: nithinraok <[email protected]>

* Fix runtime check (#4501)

* Runtime check refinements

Signed-off-by: Boris Fomitchev <[email protected]>

* Added fp32 casting for ASR nets export

Signed-off-by: Boris Fomitchev <[email protected]>

* style

Signed-off-by: Boris Fomitchev <[email protected]>

* Used torch.float32 for clarity

Signed-off-by: Boris Fomitchev <[email protected]>

* Fixing parameters passing

Signed-off-by: Boris Fomitchev <[email protected]>

* Update finetune label models (#4504)

* initial_script

Signed-off-by: nithinraok <[email protected]>

* move old script

Signed-off-by: nithinraok <[email protected]>

* remove finetune func from label models

Signed-off-by: nithinraok <[email protected]>

* style clean

Signed-off-by: nithinraok <[email protected]>

* updated config

Signed-off-by: nithinraok <[email protected]>

* update tutorial

Signed-off-by: nithinraok <[email protected]>

* lgtm fixes

Signed-off-by: nithinraok <[email protected]>

* updated based on comments

Signed-off-by: nithinraok <[email protected]>

* update doc

Signed-off-by: nithinraok <[email protected]>

* [ASR][Breaking Change] Update signature of Hypothesis alignments (#4511)

* Preserve logprobs when preserving alignments

Signed-off-by: smajumdar <[email protected]>

* Update tests for rnnt gredy and beam search

Signed-off-by: smajumdar <[email protected]>

* Update all dependents of alignments

Signed-off-by: smajumdar <[email protected]>

* Update docs

Signed-off-by: smajumdar <[email protected]>

* Weighted bucketing (#4530)

* Additional sentencepiece args - Byte fallback, split digits, split_on_whitespace (#4525)

* Fix geglu without fusion

Signed-off-by: MaximumEntropy <[email protected]>

* Add extra args

Signed-off-by: MaximumEntropy <[email protected]>

* Reset transformer

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Fix spm arg

Signed-off-by: MaximumEntropy <[email protected]>

* Fix help string

Signed-off-by: MaximumEntropy <[email protected]>

* Add support for ASR Adapter Auxiliary Losses (#4480)

* Add support for access mixin registry of custom losses

Signed-off-by: smajumdar <[email protected]>

* add support for asr custom losses

Signed-off-by: smajumdar <[email protected]>

* Update for l2 loss

Signed-off-by: smajumdar <[email protected]>

* Add unittests

Signed-off-by: smajumdar <[email protected]>

* Add unittests

Signed-off-by: smajumdar <[email protected]>

* Add unittests

Signed-off-by: smajumdar <[email protected]>

* Update registration of tensors to reset after finishing step

Signed-off-by: smajumdar <[email protected]>

* Remove comment

Signed-off-by: smajumdar <[email protected]>

* Remove comment

Signed-off-by: smajumdar <[email protected]>

* Update SSL models

Signed-off-by: smajumdar <[email protected]>

* Add support for validation step properly registering tensors

Signed-off-by: smajumdar <[email protected]>

* Move reset of registry outside

Signed-off-by: smajumdar <[email protected]>

* update (#4520)

Signed-off-by: stevehuang52 <[email protected]>

* fix duplex inference with grammars (#4517)

* fix duplex inference with grammars

Signed-off-by: ekmb <[email protected]>

* add ci test for duplex, fix electronic last sym bug

Signed-off-by: ekmb <[email protected]>

* test fix

Signed-off-by: ekmb <[email protected]>

* fix jenkins

Signed-off-by: ekmb <[email protected]>

* update jenkins grammars

Signed-off-by: ekmb <[email protected]>

* add pt to the docs

Signed-off-by: ekmb <[email protected]>

* fix jenkins

Signed-off-by: ekmb <[email protected]>

* disable test

Signed-off-by: ekmb <[email protected]>

* fix jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins refactor

Signed-off-by: ekmb <[email protected]>

* fix jenkins

Signed-off-by: ekmb <[email protected]>

* fix jenkins

Signed-off-by: ekmb <[email protected]>

* fix jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins

Signed-off-by: ekmb <[email protected]>

* jenkins

Signed-off-by: ekmb <[email protected]>

* test

Signed-off-by: ekmb <[email protected]>

* test

Signed-off-by: ekmb <[email protected]>

* test

Signed-off-by: ekmb <[email protected]>

* test

Signed-off-by: ekmb <[email protected]>

Co-authored-by: Yang Zhang <[email protected]>

* Add Bucketing support to TarredAudioToClassificationLabelDataset (#4465)

* Add Bucketing support to TarredAudioToClassificationLabelDataset

Signed-off-by: Ewald Enzinger <[email protected]>

* Add MTEncDec Finetune support (#4540)

* add FT support

Signed-off-by: Abhinav Khattar <[email protected]>

* rm preproc

Signed-off-by: Abhinav Khattar <[email protected]>

* review changes

Signed-off-by: Abhinav Khattar <[email protected]>

* add CI

Signed-off-by: Abhinav Khattar <[email protected]>

* newline fix

Signed-off-by: Abhinav Khattar <[email protected]>

* CI fix

Signed-off-by: Abhinav Khattar <[email protected]>

* clean up

Signed-off-by: Abhinav Khattar <[email protected]>

* post training cleanup

Signed-off-by: Abhinav Khattar <[email protected]>

* test

Signed-off-by: Abhinav Khattar <[email protected]>

* revert

Signed-off-by: Abhinav Khattar <[email protected]>

* CI test

Signed-off-by: Abhinav Khattar <[email protected]>

* revert CI changes

Signed-off-by: Abhinav Khattar <[email protected]>

* original CI

Signed-off-by: Abhinav Khattar <[email protected]>

Co-authored-by: Sandeep Subramanian <[email protected]>

* Add nsys profiling (#4539)

* add nsys profiling

Signed-off-by: ericharper <[email protected]>

* only access omegaconf in setup

Signed-off-by: ericharper <[email protected]>

* use robust get_rank function

Signed-off-by: ericharper <[email protected]>

* simplify

Signed-off-by: ericharper <[email protected]>

* Update megatron prompt learning interface to dialogue  (#4545)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assist…
Davood-M pushed a commit to Davood-M/NeMo that referenced this pull request Aug 9, 2022
* Dataloader, collector, loss and metric for multiscale diarization decoder  (NVIDIA#4187)

* First commit

Signed-off-by: Taejin Park <[email protected]>

* Checked funtionality and imports

Signed-off-by: Taejin Park <[email protected]>

* fixed import issues

Signed-off-by: Taejin Park <[email protected]>

* Removed the changed made by mistake

Signed-off-by: Taejin Park <[email protected]>

* Style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM errors 001

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM and style fix

Signed-off-by: Taejin Park <[email protected]>

* Changed docstrings

Signed-off-by: Taejin Park <[email protected]>

* LGTM again

Signed-off-by: Taejin Park <[email protected]>

* Removed unnecessary torch setting lines

Signed-off-by: Taejin Park <[email protected]>

* Style fix and isort

Signed-off-by: Taejin Park <[email protected]>

* jbalam-nv comments reflected

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected comments and created _diar_label.py

Signed-off-by: Taejin Park <[email protected]>

* Typo fix and style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed target_spks[0] index error

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* LGTM unused import IterDataset

Signed-off-by: Taejin Park <[email protected]>

* revert collection doc year

Signed-off-by: Taejin Park <[email protected]>

* Code format error in collections.py

Signed-off-by: Taejin Park <[email protected]>

* fix collections space format error

Signed-off-by: Taejin Park <[email protected]>

* merged main correctly

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected all comments and tested

Signed-off-by: Taejin Park <[email protected]>

* style fix and LGTM

Signed-off-by: Taejin Park <[email protected]>

* rttm_filepath to rttm_file and removed self included funcs, tested

Signed-off-by: Taejin Park <[email protected]>

Co-authored-by: Nithin Rao <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* removed references to data_dir

Signed-off-by: Matvei Novikov <[email protected]>

* added missing parameters to data preparation script

Signed-off-by: Matvei Novikov <[email protected]>

* removed unnecessary file extension check

Signed-off-by: Matvei Novikov <[email protected]>

* Add ASR CTC Decoding module (NVIDIA#4342)

* Initial commit

Signed-off-by: smajumdar <[email protected]>

* Full support for decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Temp

Signed-off-by: smajumdar <[email protected]>

* Fix labels of y_sequence

Signed-off-by: smajumdar <[email protected]>

* Set support for sentencepiece subword merging

Signed-off-by: smajumdar <[email protected]>

* Fix char and word based token merge alignment

Signed-off-by: smajumdar <[email protected]>

* Revert incorrect change

Signed-off-by: smajumdar <[email protected]>

* Update docstring

Signed-off-by: smajumdar <[email protected]>

* Improve compatibility with greedy tokens and log probs

Signed-off-by: smajumdar <[email protected]>

* Update scripts to use decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix decoding of ctc models

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Option to disable mp in VAD via num_workers=1 (NVIDIA#4317)

* Option to disable mp in VAD via num_workers=1

In certain environments python multiprocessing can deadlock. This adds a convenient version to disable by setting num_workers to 1.

Signed-off-by: Georg Kucsko <[email protected]>

* add none handling

Signed-off-by: Georg Kucsko <[email protected]>

* additional none handling

Signed-off-by: Georg Kucsko <[email protected]>

Co-authored-by: fayejf <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* remove redundant bias expand (NVIDIA#4382)

* remove redundant bias expand

Signed-off-by: Xiaowei Ren <[email protected]>

* delete redundant code

Signed-off-by: Xiaowei Ren <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* fixed style

Signed-off-by: Matvei Novikov <[email protected]>

* Add option for specifying wandb save_dir from config (NVIDIA#4379)

* give option to user to specify wandb save dir via config

Signed-off-by: Shantanu Acharya <[email protected]>

* create save_dir directory for wandb logger if not exists

Signed-off-by: Shantanu Acharya <[email protected]>

* update save_dir get method with a default value

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (NVIDIA#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* [Bugfix][TTS] wrong order of returned tuple for general_collate_fn. (NVIDIA#4388)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Merge r1.10.0 main (NVIDIA#4398)

* update branch

Signed-off-by: ericharper <[email protected]>

* Set headscale false (NVIDIA#4364)

Signed-off-by: MaximumEntropy <[email protected]>

* Add wandb as dependency (NVIDIA#4365)

Signed-off-by: smajumdar <[email protected]>

* Raise trainer error (NVIDIA#4356)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Set headscale false (NVIDIA#4364) (NVIDIA#4366)

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: smajumdar <[email protected]>

* Finetuning changes for BART (NVIDIA#4003)

* Temp

Signed-off-by: MaximumEntropy <[email protected]>

* Checkpoint converter to nemo for bart

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Make position embedding expansion specific to a batch to avoid checkpoint size mismatches (NVIDIA#4357)

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Fix logging warning

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Fix electronic bug, new time ITN rule (NVIDIA#4355)

* fix electronic bug

Signed-off-by: ekmb <[email protected]>

* add new itn time rule

Signed-off-by: ekmb <[email protected]>

* revert domain changes

Signed-off-by: ekmb <[email protected]>

* remove repetition

Signed-off-by: ekmb <[email protected]>

* Correct support for dataclasses in default module dim (NVIDIA#4372)

* Correct support for dataclasses in default module dim

Signed-off-by: smajumdar <[email protected]>

* Fix path for save of results

Signed-off-by: smajumdar <[email protected]>

* fix pad id bug (NVIDIA#4377)

Signed-off-by: Yi Dong <[email protected]>

* Question answering bug fix (NVIDIA#4381)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assistant data processsor

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins for SGDGEN local checkpoint

Signed-off-by: Zhilin Wang <[email protected]>

* update style

Signed-off-by: Zhilin Wang <[email protected]>

* use local vocab file for Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* patch for Jenkins CI using local file

Signed-off-by: Zhilin Wang <[email protected]>

* add slot filling prediction and metrics

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused code

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* refactor metrics code out of Dialogue GPT Model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate backward compatible support for IntentSlotClassificationModel (bert model)

Signed-off-by: Zhilin Wang <[email protected]>

* save prediction file for IntentSlotClassification

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue gpt model training for megatron gpt

Signed-off-by: Zhilin Wang <[email protected]>

* remove batch generate for HF GPT2, which causes lower performance

Signed-off-by: Zhilin Wang <[email protected]>

* add few shot capability to dialogue gpt model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile and remove unused import

Signed-off-by: Zhilin Wang <[email protected]>

* update code description and clarity

Signed-off-by: Zhilin Wang <[email protected]>

* address PR comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate compatibility with ZeroShotIntentModel

Signed-off-by: Zhilin Wang <[email protected]>

* rename folder to dialogue due to increased scope and further refactor for clarity

Signed-off-by: Zhilin Wang <[email protected]>

* added dialogue GPT for sequence generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* add CI test for DialogueGPTGenerationModel

Signed-off-by: Zhilin Wang <[email protected]>

* integrate DialogueS2SGenerationModel for generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* modify huggingface utils to support HF t5/BART models

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* fix bleu metric style

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* update based on PR NVIDIA#3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 2 based on PR NVIDIA#3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 3 based on PR NVIDIA#3893

Signed-off-by: Zhilin Wang <[email protected]>

* integrate sgd generation based on user user utterance and system slot-values to generate system utterance

Signed-off-by: Zhilin Wang <[email protected]>

* add validation model saving capabilities

Signed-off-by: Zhilin Wang <[email protected]>

* cleaned up code for SGD Based Answer extender

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue Generation CI

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* fix Jenkins CI issue"

Signed-off-by: Zhilin Wang <[email protected]>

* add support for design dataset

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary imports

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support megatron for dialogue_s2s_generation_model

Signed-off-by: Zhilin Wang <[email protected]>

* reduce loaded samples in MSMarcoDataProcessor to 64 when cfg.model.dataset.debug_mode=True

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update CI

Signed-off-by: Zhilin Wang <[email protected]>

* update checkpoint and predictions filename to include epoch number

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate HF BART MNLI into zero shot intent model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Nearest Neighbour Model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* refactor Dialogue SGD Data Processor to make interface for models cleaner

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue S2S Generation model for DialogueSGDDataProcessor interface

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support sgd and drive thru datasets by zero shot model and nearest neighbour model

Signed-off-by: Zhilin Wang <[email protected]>

* add prediction saving code to nearest neighbour and zero shot intent models

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo in sgd data processor

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Mellon QA Data Processor

Signed-off-by: Zhilin Wang <[email protected]>

* update mellon qa

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue.py to remove outdated info

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* add dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* address review comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix for cfg

Signed-off-by: Zhilin Wang <[email protected]>

* make dependency on apex optional

Signed-off-by: Zhilin Wang <[email protected]>

* change NLPDDPluggin calling logic to make it possible to run without apex

Signed-off-by: Zhilin Wang <[email protected]>

* add first draft of tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* reduce ms marco size by removing lines without wellFormedAnswers

Signed-off-by: Zhilin Wang <[email protected]>

* address pr comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update colab tutorial link in dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* include unit test and some refactor to facilitate unit test

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* address pr issues

Signed-off-by: Zhilin Wang <[email protected]>

* remove typos in dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* support larger files for question answering

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary artifacts to reduce memory use

Signed-off-by: Zhilin Wang <[email protected]>

* put 0 tensor to device

Signed-off-by: Zhilin Wang <[email protected]>

* update link within dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously delete files

Signed-off-by: Zhilin Wang <[email protected]>

* update error handling when loss = nan

Signed-off-by: Zhilin Wang <[email protected]>

* update nan handling

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss func

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss

Signed-off-by: Zhilin Wang <[email protected]>

* fix type error raised in qa_dataset.py

Signed-off-by: Zhilin Wang <[email protected]>

* add error checking message

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update exp logging

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* limit number of negative samples

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused methods and style fix

Signed-off-by: Zhilin Wang <[email protected]>

* add more documentation

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* changes base on PR review

Signed-off-by: Zhilin Wang <[email protected]>

* set wandb logger falseby default

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

* style fix

* correct typo

* style fix

* style fix

Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>

* Fix ASR Typos in tutorials (NVIDIA#4384)

* Fix typos

Signed-off-by: smajumdar <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (NVIDIA#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
(cherry picked from commit 0322b15)

Co-authored-by: Travis Bartley <[email protected]>

* Add Docs for NeMo Adapters (NVIDIA#4369)

Signed-off-by: smajumdar <[email protected]>

* Update NeMo docs (NVIDIA#4397)

Signed-off-by: smajumdar <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* update branch

Signed-off-by: ericharper <[email protected]>

* remove Copy of

Signed-off-by: ericharper <[email protected]>

Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Micha Livne <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Travis Bartley <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* [bugfix][TTS] pitch, voiced_mask, prob_voiced have the same values. (NVIDIA#4392)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Fixing import error in some cases (NVIDIA#4401)

Signed-off-by: Boris Fomitchev <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Fixing bugs in calling method ctc_decoder_predictions_tensor. (NVIDIA#4414)

* updated ctc decoding calls.

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Update with new conformer checkpoints. (NVIDIA#4417)

Signed-off-by: Matvei Novikov <[email protected]>

* [TTS] add static method decorator. (NVIDIA#4443)

* [TTS] add static method decorator.

Signed-off-by: Xuesong Yang <[email protected]>

* remove protect prefix

Signed-off-by: Xuesong Yang <[email protected]>

* fixed style error

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

Co-authored-by: Taejin Park <[email protected]>
Co-authored-by: Nithin Rao <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Georg Kucsko <[email protected]>
Co-authored-by: fayejf <[email protected]>
Co-authored-by: Xiaowei Ren <[email protected]>
Co-authored-by: Shantanu Acharya <[email protected]>
Co-authored-by: Travis Bartley <[email protected]>
Co-authored-by: tbartley94 <[email protected]>
Co-authored-by: Xuesong Yang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Micha Livne <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Boris Fomitchev <[email protected]>
Co-authored-by: Vahid Noroozi <[email protected]>
Signed-off-by: David Mosallanezhad <[email protected]>
fayejf added a commit that referenced this pull request Aug 23, 2022
* stateless RNNT working

Signed-off-by: Hainan Xu <[email protected]>

* batch decode working

Signed-off-by: Hainan Xu <[email protected]>

* working backup

Signed-off-by: Hainan Xu <[email protected]>

* good working version

Signed-off-by: Hainan Xu <[email protected]>

* temporarily make norm layer have affine

Signed-off-by: Hainan Xu <[email protected]>

* temp

Signed-off-by: Hainan Xu <[email protected]>

* temp

Signed-off-by: Hainan Xu <[email protected]>

* [TTS] add staticmethod decoration for BetaBinomialInterpolator (#4319)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* [TTS] remove redundant lines and declare global variables and capture (#4320)

exception of non-supported windows.

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Finetune T5 on the prefix-lm objective (#4328)

* Add script and yaml config

Signed-off-by: MaximumEntropy <[email protected]>

* Fix yaml config

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Update yaml to remove hardcoded model path

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Oleksii Kuchaiev <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Fuse bias with geglu in ParallelMLP (#4213)

* add code of fused_bias_geglu

* call fused_bias_geglu in ParallelMLP

* fix some bugs

* change biad_gelu_activation to bias_activation_fusion

* fix the setting of bias_actication_fusion for T5

* delete bias_gelu_fusion from T5 example config

* push reformatted files

* hto4h gemms fusion

* remove hto4h gemms fusion

* push reformatted files

* disable bias_activation_fusion while activation is not geglu

* add bias_activation_fusion in yaml config file

* add bias_gelu_fusion in T5 config yaml file to pass CI test

* change bias_gelu_fusion to bias_activation_fusion for T5 CI test

* recover latest change

Co-authored-by: Sandeep Subramanian <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Support larger datasets for question answering  (#4205)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assistant data processsor

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins for SGDGEN local checkpoint

Signed-off-by: Zhilin Wang <[email protected]>

* update style

Signed-off-by: Zhilin Wang <[email protected]>

* use local vocab file for Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* patch for Jenkins CI using local file

Signed-off-by: Zhilin Wang <[email protected]>

* add slot filling prediction and metrics

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused code

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* refactor metrics code out of Dialogue GPT Model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate backward compatible support for IntentSlotClassificationModel (bert model)

Signed-off-by: Zhilin Wang <[email protected]>

* save prediction file for IntentSlotClassification

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue gpt model training for megatron gpt

Signed-off-by: Zhilin Wang <[email protected]>

* remove batch generate for HF GPT2, which causes lower performance

Signed-off-by: Zhilin Wang <[email protected]>

* add few shot capability to dialogue gpt model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile and remove unused import

Signed-off-by: Zhilin Wang <[email protected]>

* update code description and clarity

Signed-off-by: Zhilin Wang <[email protected]>

* address PR comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate compatibility with ZeroShotIntentModel

Signed-off-by: Zhilin Wang <[email protected]>

* rename folder to dialogue due to increased scope and further refactor for clarity

Signed-off-by: Zhilin Wang <[email protected]>

* added dialogue GPT for sequence generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* add CI test for DialogueGPTGenerationModel

Signed-off-by: Zhilin Wang <[email protected]>

* integrate DialogueS2SGenerationModel for generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* modify huggingface utils to support HF t5/BART models

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* fix bleu metric style

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* update based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 2 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 3 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* integrate sgd generation based on user user utterance and system slot-values to generate system utterance

Signed-off-by: Zhilin Wang <[email protected]>

* add validation model saving capabilities

Signed-off-by: Zhilin Wang <[email protected]>

* cleaned up code for SGD Based Answer extender

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue Generation CI

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* fix Jenkins CI issue"

Signed-off-by: Zhilin Wang <[email protected]>

* add support for design dataset

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary imports

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support megatron for dialogue_s2s_generation_model

Signed-off-by: Zhilin Wang <[email protected]>

* reduce loaded samples in MSMarcoDataProcessor to 64 when cfg.model.dataset.debug_mode=True

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update CI

Signed-off-by: Zhilin Wang <[email protected]>

* update checkpoint and predictions filename to include epoch number

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate HF BART MNLI into zero shot intent model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Nearest Neighbour Model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* refactor Dialogue SGD Data Processor to make interface for models cleaner

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue S2S Generation model for DialogueSGDDataProcessor interface

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support sgd and drive thru datasets by zero shot model and nearest neighbour model

Signed-off-by: Zhilin Wang <[email protected]>

* add prediction saving code to nearest neighbour and zero shot intent models

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo in sgd data processor

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Mellon QA Data Processor

Signed-off-by: Zhilin Wang <[email protected]>

* update mellon qa

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue.py to remove outdated info

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* add dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* address review comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix for cfg

Signed-off-by: Zhilin Wang <[email protected]>

* make dependency on apex optional

Signed-off-by: Zhilin Wang <[email protected]>

* change NLPDDPluggin calling logic to make it possible to run without apex

Signed-off-by: Zhilin Wang <[email protected]>

* add first draft of tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* reduce ms marco size by removing lines without wellFormedAnswers

Signed-off-by: Zhilin Wang <[email protected]>

* address pr comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update colab tutorial link in dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* include unit test and some refactor to facilitate unit test

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* address pr issues

Signed-off-by: Zhilin Wang <[email protected]>

* remove typos in dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* support larger files for question answering

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary artifacts to reduce memory use

Signed-off-by: Zhilin Wang <[email protected]>

* put 0 tensor to device

Signed-off-by: Zhilin Wang <[email protected]>

* update link within dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously delete files

Signed-off-by: Zhilin Wang <[email protected]>

* update error handling when loss = nan

Signed-off-by: Zhilin Wang <[email protected]>

* update nan handling

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss func

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss

Signed-off-by: Zhilin Wang <[email protected]>

* fix type error raised in qa_dataset.py

Signed-off-by: Zhilin Wang <[email protected]>

* add error checking message

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update exp logging

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* limit number of negative samples

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused methods and style fix

Signed-off-by: Zhilin Wang <[email protected]>

* add more documentation

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* changes base on PR review

Signed-off-by: Zhilin Wang <[email protected]>

Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Fix bugs in indexed dataset exam script (#4325)

* fix the typo

Signed-off-by: Yi Dong <[email protected]>

* add neighbors option

Signed-off-by: Yi Dong <[email protected]>

* change the argument name

Signed-off-by: Yi Dong <[email protected]>

Co-authored-by: Micha Livne <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Adding docs for ASR SSL (#4303)

* Initial commit for SSL docs

Signed-off-by: Krishna Puvvada <[email protected]>

* ssl docs update-1

Signed-off-by: Krishna Puvvada <[email protected]>

* ssl docs update-2

Signed-off-by: Krishna Puvvada <[email protected]>

Co-authored-by: Krishna Puvvada <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Fuse grad division into async grad allreduce (#4327)

* O2 runs but O1 does not

Signed-off-by: ericharper <[email protected]>

* disable async for O1

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

* update async flag in configure_optimizers

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

* revert

Signed-off-by: ericharper <[email protected]>

* update _require if using async

Signed-off-by: ericharper <[email protected]>

* clean comments

Signed-off-by: ericharper <[email protected]>

* always all_reduce

Signed-off-by: ericharper <[email protected]>

* add async grad allreduce and chunk optimization to T5

* push reformatted files after style check

* set chunk size as 0 while async grad allreduce is off

* more experiments show that 125MB is a better default chunk size for most cases

* add grad_allreduce_chunk_size_mb for GPT-3

* at the end of each training step, wait until all async grad allreduce works are done

* replace individual allreduce work.wait() with a single dGPU evice synchroonization

* add code of fused_bias_geglu

* call fused_bias_geglu in ParallelMLP

* record the status of each allreduce work seems too much for perf

* add more comments

* push a reformatted file

* fix some bugs

* change biad_gelu_activation to bias_activation_fusion

* fix the setting of bias_actication_fusion for T5

* delete bias_gelu_fusion from T5 example config

* push reformatted files

* fuse grad scale with allreduce

* push reformatted files

* hto4h gemms fusion

* remove hto4h gemms fusion

* add grad_scale_ar_fusion into GPT-3

* push reformatted files

* push reformatted files

* rename grad_scale_ar_fusion to grad_div_ar_fusion

* disable bias_activation_fusion while activation is not geglu

* add bias_activation_fusion in yaml config file

* add bias_gelu_fusion in T5 config yaml file to pass CI test

* change bias_gelu_fusion to bias_activation_fusion for T5 CI test

* recover latest change

* add grad_div_ar_fusion in config yaml file

* remove a redundant float()

Co-authored-by: ericharper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Update container to 22.05 (#4329)

* update container to 22.05

Signed-off-by: ericharper <[email protected]>

* try adding safe directory

Signed-off-by: ericharper <[email protected]>

* try env var

Signed-off-by: ericharper <[email protected]>

* printenv

Signed-off-by: ericharper <[email protected]>

* try GIT_BRANCH

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

* remove dbug statements

Signed-off-by: ericharper <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Torchaudio installation fix (#4330)

* separate installer added

Signed-off-by: Aleksandr Laptev <[email protected]>

* apply suggestions, minor fixes

Signed-off-by: Aleksandr Laptev <[email protected]>

Co-authored-by: Aleksandr Laptev <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* [TTS] enforced pin_memory = True (#4341)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Merge r1.9.0 main (#4331)

* update branch

Signed-off-by: ericharper <[email protected]>

* update package info

Signed-off-by: ericharper <[email protected]>

* cleaned up TN/ ITN doc (#4119)

* cleaned up TN/ ITN doc

Signed-off-by: Yang Zhang <[email protected]>

* fix typo

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* Draft: Fix restoring from checkpoint for case when `model.common_dataset_parameters.label_vocab_dir` is provided (#4136)

* Fix restoring from checkpoint with label vocab dir

Signed-off-by: PeganovAnton <[email protected]>

* Add tests for various ways to pass label ids to model

Signed-off-by: PeganovAnton <[email protected]>

* Fix typo

Signed-off-by: PeganovAnton <[email protected]>

* Fix typo

Signed-off-by: PeganovAnton <[email protected]>

* Do not create tmp directory

Signed-off-by: PeganovAnton <[email protected]>

* Fix parameter name

Signed-off-by: PeganovAnton <[email protected]>

* finish cherry-pick op

Signed-off-by: PeganovAnton <[email protected]>

* Fix labels errors

Signed-off-by: PeganovAnton <[email protected]>

* Remove duplicate stage

Signed-off-by: PeganovAnton <[email protected]>

* Change target branch

Signed-off-by: PeganovAnton <[email protected]>

* fix doc (#4146)

Signed-off-by: Yang Zhang <[email protected]>

* Tacotron2 retrain (#4103)

* fix yaml

Signed-off-by: treacker <[email protected]>

* Fix for new TTSDataset class

Signed-off-by: treacker <[email protected]>

* added wandb logging

Signed-off-by: treacker <[email protected]>

* added wandb logging

Signed-off-by: treacker <[email protected]>

* fix numpy version

Signed-off-by: treacker <[email protected]>

* fix numpy version

Signed-off-by: treacker <[email protected]>

* inference fix

Signed-off-by: treacker <[email protected]>

* removed old code

Signed-off-by: treacker <[email protected]>

* updated parser logic

Signed-off-by: treacker <[email protected]>

* reverted version update

Signed-off-by: treacker <[email protected]>

* refactored parser logic

Signed-off-by: treacker <[email protected]>

* Updated Jenkinsfile

Signed-off-by: treacker <[email protected]>

* Refactored tutorial for Tacotron2

Signed-off-by: treacker <[email protected]>

* Made backward compatibility

Signed-off-by: treacker <[email protected]>

* Made backward compatibility

Signed-off-by: treacker <[email protected]>

* Update Jenkinsfile

Signed-off-by: treacker <[email protected]>

* Update tacotron.yaml

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* cleaned up TN/ ITN doc (#4119)

* cleaned up TN/ ITN doc

Signed-off-by: Yang Zhang <[email protected]>

* fix typo

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>
Signed-off-by: treacker <[email protected]>

* Check implicit grad acc in GLUE dataset building (#4123)

* Check implicit grad acc in GLUE dataset building

Signed-off-by: MaximumEntropy <[email protected]>

* Fix jenkins test for GLUE/XNLI

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Fixed jenkins

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>

* Multiprocess improvements (#4127)

* initial commit

Signed-off-by: nithinraok <[email protected]>

* start fix

Signed-off-by: nithinraok <[email protected]>

* improve multiprocessing speed while creating speaker dataset

Signed-off-by: nithinraok <[email protected]>

* updated scp to filelist

Signed-off-by: nithinraok <[email protected]>

* notebooks' link, typo and import  fix  (#4158)

* redo missing pr 4007

Signed-off-by: fayejf <[email protected]>

* remove extremely unreliable links

Signed-off-by: fayejf <[email protected]>

* update speaker docs (#4164)

* update speaker docs

Signed-off-by: nithinraok <[email protected]>

* chunks -> segments

Signed-off-by: nithinraok <[email protected]>

* Khz -> kHz

Signed-off-by: nithinraok <[email protected]>

* small fix (#4180)

Signed-off-by: fayejf <[email protected]>

* fix the server key value problem (#4196)

Signed-off-by: Yi Dong <[email protected]>

* Fix/punctuation/trainer required for setting test data (#4199)

* Draft of fix

Signed-off-by: PeganovAnton <[email protected]>

* Add warnings and replace globa_step with current_epoch

Signed-off-by: PeganovAnton <[email protected]>

* Small improvements to warnings

Signed-off-by: PeganovAnton <[email protected]>

* Error and warning messages improvements

Signed-off-by: PeganovAnton <[email protected]>

* Replace self.trainer with self._trainer

Signed-off-by: PeganovAnton <[email protected]>

* Update ContextNet version (#4207)

Signed-off-by: smajumdar <[email protected]>

* fix bugs for dialogue tutorial (#4211)

Signed-off-by: Zhilin Wang <[email protected]>

* Dialogue tutorial fix (#4214)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* Add docs for Thutmose Tagger (#4173)

* Add docs for Thutmose Tagger

Signed-off-by: Alexandra Antonova <[email protected]>

* add level in docs

Signed-off-by: Alexandra Antonova <[email protected]>

* delete folder to avoid error with running when folder exists from previous run

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>
Co-authored-by: ekmb <[email protected]>

* Dialogue tutorial fix (#4218)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously deleted files

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* Dialogue tutorial fix (#4221)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously deleted files

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* fix syntax error in ipynb-file (#4228)

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>

* fix json serialize (#4235)

Signed-off-by: Yi Dong <[email protected]>

* Prompt Learning Typo Fixes (#4238)

* Prompt tuning notebook typo fixes

Signed-off-by: Virginia Adams <[email protected]>

* Update tutorials.rst

* Update prompt_learning.rst

* Update prompt_learning.rst

* fixing bug 3642622 (#4250)

* fixing bug 3642622

Signed-off-by: Ghasem Pasandi <[email protected]>

* fixing bug 3642622

Signed-off-by: Ghasem Pasandi <[email protected]>

Co-authored-by: Ghasem Pasandi <[email protected]>

* fix broken link in the tutorial (#4257)

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>

* Typo fix, branch change, better download messagae (#4262)

Signed-off-by: Virginia Adams <[email protected]>

* Raise error if bicleaner is not installed in NMT Data preprocesing notebook (#4264)

* Raise error if bicleaner is not installed

Signed-off-by: MaximumEntropy <[email protected]>

* Clear cells

Signed-off-by: MaximumEntropy <[email protected]>

* Fix missing validation dataset, whitelist certain keywords for datasets (#4269)

* Fix missing validation dataset, whitelist certain keywords for datasets

Signed-off-by: smajumdar <[email protected]>

* Fix missing validation dataset, whitelist certain keywords for datasets

Signed-off-by: smajumdar <[email protected]>

* Update asr configs with num_workers and pin_memory (#4270)

Signed-off-by: smajumdar <[email protected]>

* Fix epoch end (#4265)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* Set Save on train end to false (#4274)

* Set Save on train end to false

Signed-off-by: Virginia Adams <[email protected]>

* Update prompt_learning.rst

* Update prompt_learning.rst

* Update YAML (#4261)

Signed-off-by: MaximumEntropy <[email protected]>

* Updated config to fix CI test OOM error (#4279)

* Updated config to fix CI test issue

Signed-off-by: Virginia Adams <[email protected]>

* Increased num workers

Signed-off-by: Virginia Adams <[email protected]>

* verbose k2 install, skip if failed (#4289)

Signed-off-by: Aleksandr Laptev <[email protected]>

Co-authored-by: Aleksandr Laptev <[email protected]>

* Changed total virtual prompt tokens (#4295)

* Changed total virtual prompt tokens

Signed-off-by: Virginia Adams <[email protected]>

* put number of workers back

Signed-off-by: Virginia Adams <[email protected]>

* upper bound lightning

Signed-off-by: ericharper <[email protected]>

* update branch

Signed-off-by: ericharper <[email protected]>

* update config

Signed-off-by: ericharper <[email protected]>

* remove duplicate test

Signed-off-by: ericharper <[email protected]>

* fix tn test cases

Signed-off-by: ericharper <[email protected]>

* add another safe.directory

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: PeganovAnton <[email protected]>
Co-authored-by: treacker <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Nithin Rao <[email protected]>
Co-authored-by: fayejf <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: bene-ges <[email protected]>
Co-authored-by: Alexandra Antonova <[email protected]>
Co-authored-by: ekmb <[email protected]>
Co-authored-by: Virginia Adams <[email protected]>
Co-authored-by: Ghasem <[email protected]>
Co-authored-by: Ghasem Pasandi <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* [TTS] Training Fastpitch on German text and phonemes and finetuning HiFi-GAN on predicted mels (#4266)

* initial commit

Signed-off-by: Akshit Arora <[email protected]>

* cleared notebook outputs

Signed-off-by: Akshit Arora <[email protected]>

* formatting errors

Signed-off-by: Akshit Arora <[email protected]>

* formatting

Signed-off-by: Akshit Arora <[email protected]>

* addressed comments

Signed-off-by: Akshit Arora <[email protected]>

* addressed comments on tutorial

Signed-off-by: Akshit Arora <[email protected]>

* updated tutorial

Signed-off-by: Akshit Arora <[email protected]>

* updated grammar and fastpitch description

Signed-off-by: Akshit Arora <[email protected]>

* updated with feedback

Signed-off-by: Akshit Arora <[email protected]>

* updated with feedback

Signed-off-by: Akshit Arora <[email protected]>

* removed unused imports

Signed-off-by: Akshit Arora <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Speedup the speech commands dataset processing script (#4347)

* Add multiprocessing support to the google speech commands dataset processing script

Signed-off-by: Shantanu Acharya <[email protected]>

* fix number of args error with __extract_all_files function

Signed-off-by: Shantanu Acharya <[email protected]>

* fix styling issues

Signed-off-by: Shantanu Acharya <[email protected]>

* fix bugs with silence set construction and update librosa output write to use soundfile write

Signed-off-by: Shantanu Acharya <[email protected]>

* add docstrings and return values in __construct_filepaths as dictionary

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* fix wrong requirement (#4349)

Signed-off-by: Yang Zhang <[email protected]>

Co-authored-by: Sandeep Subramanian <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Refactored path to manifest (#4251)

Signed-off-by: Evgeniy Shabalin <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* IPA support for TTS (#4310)

* IPA tokenizer and G2P untested draft

Signed-off-by: Jocelyn Huang <[email protected]>

* Add IPA CMUdict and new heteronyms list

Signed-off-by: Jocelyn Huang <[email protected]>

* Add draft FastPitch IPA config

Signed-off-by: Jocelyn Huang <[email protected]>

* Minor bugfixes for IPA training

Signed-off-by: Jocelyn Huang <[email protected]>

* Add phoneme_probability to IPA G2P

Signed-off-by: Jocelyn Huang <[email protected]>

* Updates to IPA FastPitch training config

Signed-off-by: Jocelyn Huang <[email protected]>

* Update IPA dict and heteronyms file

Signed-off-by: Jocelyn Huang <[email protected]>

* Adjust default lr for IPA FastPitch to 1e-3

Signed-off-by: Jocelyn Huang <[email protected]>

* Rename IPA CMUdict to reflect date

Signed-off-by: Jocelyn Huang <[email protected]>

* Add docstrings for IPA tokenizer and G2P, update CMUdict path for config

Signed-off-by: Jocelyn Huang <[email protected]>

* Fix IPA vocab ordering, add options to uppercase graphemes and remove stress symbols

Signed-off-by: Jocelyn Huang <[email protected]>

* Mark IPA classes as experimental

Signed-off-by: Jocelyn Huang <[email protected]>

* Update apostrophe-S cases

Signed-off-by: Jocelyn Huang <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Tn install (#4055)

* remove conda pynini requirement

Signed-off-by: Yang Zhang <[email protected]>

* remove remnants

Signed-off-by: Yang Zhang <[email protected]>

* merge with main

Signed-off-by: Yang Zhang <[email protected]>

* removing nlp collection dependency from text processing and thus breaking cyclyc imports

Signed-off-by: Yang Zhang <[email protected]>

* fix

Signed-off-by: Yang Zhang <[email protected]>

* fix wrong requirement

Signed-off-by: Yang Zhang <[email protected]>

* fix bug in vi

Signed-off-by: Yang Zhang <[email protected]>

* update jenkins folders

Signed-off-by: ekmb <[email protected]>

Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: ekmb <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* fix tutorial (#4352)

Signed-off-by: stevehuang52 <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* fix the post ln (#4350)

Signed-off-by: Yi Dong <[email protected]>

Co-authored-by: Eric Harper <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* [Fix] Hanging for Fully Randomized Bucketing (#4348)

* Update container to 22.05 (#4329)

* update container to 22.05

Signed-off-by: ericharper <[email protected]>

* try adding safe directory

Signed-off-by: ericharper <[email protected]>

* try env var

Signed-off-by: ericharper <[email protected]>

* printenv

Signed-off-by: ericharper <[email protected]>

* try GIT_BRANCH

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

* remove dbug statements

Signed-off-by: ericharper <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>

* Merge r1.9.0 main (#4331)

* update branch

Signed-off-by: ericharper <[email protected]>

* update package info

Signed-off-by: ericharper <[email protected]>

* cleaned up TN/ ITN doc (#4119)

* cleaned up TN/ ITN doc

Signed-off-by: Yang Zhang <[email protected]>

* fix typo

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* Draft: Fix restoring from checkpoint for case when `model.common_dataset_parameters.label_vocab_dir` is provided (#4136)

* Fix restoring from checkpoint with label vocab dir

Signed-off-by: PeganovAnton <[email protected]>

* Add tests for various ways to pass label ids to model

Signed-off-by: PeganovAnton <[email protected]>

* Fix typo

Signed-off-by: PeganovAnton <[email protected]>

* Fix typo

Signed-off-by: PeganovAnton <[email protected]>

* Do not create tmp directory

Signed-off-by: PeganovAnton <[email protected]>

* Fix parameter name

Signed-off-by: PeganovAnton <[email protected]>

* finish cherry-pick op

Signed-off-by: PeganovAnton <[email protected]>

* Fix labels errors

Signed-off-by: PeganovAnton <[email protected]>

* Remove duplicate stage

Signed-off-by: PeganovAnton <[email protected]>

* Change target branch

Signed-off-by: PeganovAnton <[email protected]>

* fix doc (#4146)

Signed-off-by: Yang Zhang <[email protected]>

* Tacotron2 retrain (#4103)

* fix yaml

Signed-off-by: treacker <[email protected]>

* Fix for new TTSDataset class

Signed-off-by: treacker <[email protected]>

* added wandb logging

Signed-off-by: treacker <[email protected]>

* added wandb logging

Signed-off-by: treacker <[email protected]>

* fix numpy version

Signed-off-by: treacker <[email protected]>

* fix numpy version

Signed-off-by: treacker <[email protected]>

* inference fix

Signed-off-by: treacker <[email protected]>

* removed old code

Signed-off-by: treacker <[email protected]>

* updated parser logic

Signed-off-by: treacker <[email protected]>

* reverted version update

Signed-off-by: treacker <[email protected]>

* refactored parser logic

Signed-off-by: treacker <[email protected]>

* Updated Jenkinsfile

Signed-off-by: treacker <[email protected]>

* Refactored tutorial for Tacotron2

Signed-off-by: treacker <[email protected]>

* Made backward compatibility

Signed-off-by: treacker <[email protected]>

* Made backward compatibility

Signed-off-by: treacker <[email protected]>

* Update Jenkinsfile

Signed-off-by: treacker <[email protected]>

* Update tacotron.yaml

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* cleaned up TN/ ITN doc (#4119)

* cleaned up TN/ ITN doc

Signed-off-by: Yang Zhang <[email protected]>

* fix typo

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>
Signed-off-by: treacker <[email protected]>

* Check implicit grad acc in GLUE dataset building (#4123)

* Check implicit grad acc in GLUE dataset building

Signed-off-by: MaximumEntropy <[email protected]>

* Fix jenkins test for GLUE/XNLI

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Fixed jenkins

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>

* Multiprocess improvements (#4127)

* initial commit

Signed-off-by: nithinraok <[email protected]>

* start fix

Signed-off-by: nithinraok <[email protected]>

* improve multiprocessing speed while creating speaker dataset

Signed-off-by: nithinraok <[email protected]>

* updated scp to filelist

Signed-off-by: nithinraok <[email protected]>

* notebooks' link, typo and import  fix  (#4158)

* redo missing pr 4007

Signed-off-by: fayejf <[email protected]>

* remove extremely unreliable links

Signed-off-by: fayejf <[email protected]>

* update speaker docs (#4164)

* update speaker docs

Signed-off-by: nithinraok <[email protected]>

* chunks -> segments

Signed-off-by: nithinraok <[email protected]>

* Khz -> kHz

Signed-off-by: nithinraok <[email protected]>

* small fix (#4180)

Signed-off-by: fayejf <[email protected]>

* fix the server key value problem (#4196)

Signed-off-by: Yi Dong <[email protected]>

* Fix/punctuation/trainer required for setting test data (#4199)

* Draft of fix

Signed-off-by: PeganovAnton <[email protected]>

* Add warnings and replace globa_step with current_epoch

Signed-off-by: PeganovAnton <[email protected]>

* Small improvements to warnings

Signed-off-by: PeganovAnton <[email protected]>

* Error and warning messages improvements

Signed-off-by: PeganovAnton <[email protected]>

* Replace self.trainer with self._trainer

Signed-off-by: PeganovAnton <[email protected]>

* Update ContextNet version (#4207)

Signed-off-by: smajumdar <[email protected]>

* fix bugs for dialogue tutorial (#4211)

Signed-off-by: Zhilin Wang <[email protected]>

* Dialogue tutorial fix (#4214)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* Add docs for Thutmose Tagger (#4173)

* Add docs for Thutmose Tagger

Signed-off-by: Alexandra Antonova <[email protected]>

* add level in docs

Signed-off-by: Alexandra Antonova <[email protected]>

* delete folder to avoid error with running when folder exists from previous run

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>
Co-authored-by: ekmb <[email protected]>

* Dialogue tutorial fix (#4218)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously deleted files

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* Dialogue tutorial fix (#4221)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously deleted files

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* fix syntax error in ipynb-file (#4228)

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>

* fix json serialize (#4235)

Signed-off-by: Yi Dong <[email protected]>

* Prompt Learning Typo Fixes (#4238)

* Prompt tuning notebook typo fixes

Signed-off-by: Virginia Adams <[email protected]>

* Update tutorials.rst

* Update prompt_learning.rst

* Update prompt_learning.rst

* fixing bug 3642622 (#4250)

* fixing bug 3642622

Signed-off-by: Ghasem Pasandi <[email protected]>

* fixing bug 3642622

Signed-off-by: Ghasem Pasandi <[email protected]>

Co-authored-by: Ghasem Pasandi <[email protected]>

* fix broken link in the tutorial (#4257)

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>

* Typo fix, branch change, better download messagae (#4262)

Signed-off-by: Virginia Adams <[email protected]>

* Raise error if bicleaner is not installed in NMT Data preprocesing notebook (#4264)

* Raise error if bicleaner is not installed

Signed-off-by: MaximumEntropy <[email protected]>

* Clear cells

Signed-off-by: MaximumEntropy <[email protected]>

* Fix missing validation dataset, whitelist certain keywords for datasets (#4269)

* Fix missing validation dataset, whitelist certain keywords for datasets

Signed-off-by: smajumdar <[email protected]>

* Fix missing validation dataset, whitelist certain keywords for datasets

Signed-off-by: smajumdar <[email protected]>

* Update asr configs with num_workers and pin_memory (#4270)

Signed-off-by: smajumdar <[email protected]>

* Fix epoch end (#4265)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* Set Save on train end to false (#4274)

* Set Save on train end to false

Signed-off-by: Virginia Adams <[email protected]>

* Update prompt_learning.rst

* Update prompt_learning.rst

* Update YAML (#4261)

Signed-off-by: MaximumEntropy <[email protected]>

* Updated config to fix CI test OOM error (#4279)

* Updated config to fix CI test issue

Signed-off-by: Virginia Adams <[email protected]>

* Increased num workers

Signed-off-by: Virginia Adams <[email protected]>

* verbose k2 install, skip if failed (#4289)

Signed-off-by: Aleksandr Laptev <[email protected]>

Co-authored-by: Aleksandr Laptev <[email protected]>

* Changed total virtual prompt tokens (#4295)

* Changed total virtual prompt tokens

Signed-off-by: Virginia Adams <[email protected]>

* put number of workers back

Signed-off-by: Virginia Adams <[email protected]>

* upper bound lightning

Signed-off-by: ericharper <[email protected]>

* update branch

Signed-off-by: ericharper <[email protected]>

* update config

Signed-off-by: ericharper <[email protected]>

* remove duplicate test

Signed-off-by: ericharper <[email protected]>

* fix tn test cases

Signed-off-by: ericharper <[email protected]>

* add another safe.directory

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: PeganovAnton <[email protected]>
Co-authored-by: treacker <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Nithin Rao <[email protected]>
Co-authored-by: fayejf <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: bene-ges <[email protected]>
Co-authored-by: Alexandra Antonova <[email protected]>
Co-authored-by: ekmb <[email protected]>
Co-authored-by: Virginia Adams <[email protected]>
Co-authored-by: Ghasem <[email protected]>
Co-authored-by: Ghasem Pasandi <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>

* fix full_randn bucket hang

Signed-off-by: stevehuang52 <[email protected]>

* remove unused variables

Signed-off-by: stevehuang52 <[email protected]>

Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: PeganovAnton <[email protected]>
Co-authored-by: treacker <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Nithin Rao <[email protected]>
Co-authored-by: fayejf <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: bene-ges <[email protected]>
Co-authored-by: Alexandra Antonova <[email protected]>
Co-authored-by: ekmb <[email protected]>
Co-authored-by: Virginia Adams <[email protected]>
Co-authored-by: Ghasem <[email protected]>
Co-authored-by: Ghasem Pasandi <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Bits of RADTTS support (#4343)

* Bits of RADTTS support

Signed-off-by: Boris Fomitchev <[email protected]>

* Fixed args mismatch

Signed-off-by: Boris Fomitchev <[email protected]>

* Style

Signed-off-by: Boris Fomitchev <[email protected]>

* Addressed review comments

Signed-off-by: Boris Fomitchev <[email protected]>

* More review comments

Signed-off-by: Boris Fomitchev <[email protected]>

Co-authored-by: Eric Harper <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Prompt Learning Pipeline Parallel (#4291)

* Added get_forward_output_and_loss_func and updated train/val steps

Signed-off-by: Virginia Adams <[email protected]>

* Added preprocess flag before prompt table/encoder access

Signed-off-by: Virginia Adams <[email protected]>

* Made two optimizer groups, one for frozen, one for soft prompt

Signed-off-by: Virginia Adams <[email protected]>

* Pipeline parallel working

Signed-off-by: Virginia Adams <[email protected]>

* Still figuring out setting lr/sched for one param group

Signed-off-by: Virginia Adams <[email protected]>

* Set betas to zero

Signed-off-by: Virginia Adams <[email protected]>

* Only unfreeze one sublayer with lr 0.0

Signed-off-by: Virginia Adams <[email protected]>

* Pipeline parallel working w/ one optimizer

Signed-off-by: Virginia Adams <[email protected]>

* Trying to fix Jenkins file

Signed-off-by: Virginia Adams <[email protected]>

* Trying to fix Jenkins file

Signed-off-by: Virginia Adams <[email protected]>

* Getting updated jenkins test to work

Signed-off-by: Virginia Adams <[email protected]>

* Getting updated jenkins test to work

Signed-off-by: Virginia Adams <[email protected]>

* added prompt learning tp and pp CI tests

Signed-off-by: Virginia Adams <[email protected]>

* Added amp_o2 model compatibility

Signed-off-by: Virginia Adams <[email protected]>

* Made CI test smaller

Signed-off-by: Virginia Adams <[email protected]>

* Still trying to get Jenkins to work

Signed-off-by: Virginia Adams <[email protected]>

* Still trying to get Jenkins to work

Signed-off-by: Virginia Adams <[email protected]>

* Temporarily moving prompt learning CI test to beginning

Signed-off-by: Virginia Adams <[email protected]>

* Changing the layer being unfrozen

Signed-off-by: Virginia Adams <[email protected]>

* debug jenkins

Signed-off-by: Virginia Adams <[email protected]>

* Move pp unfreeze to init

Signed-off-by: Virginia Adams <[email protected]>

* Try to make Jenkins test parallel

Signed-off-by: Virginia Adams <[email protected]>

* Fix python formatting

Signed-off-by: Virginia Adams <[email protected]>

* Moved prompt learning tests back to where they belong

Signed-off-by: Virginia Adams <[email protected]>

* add back checkpoint convertion CI test

Signed-off-by: Virginia Adams <[email protected]>

* Revert "add back checkpoint convertion CI test"

This reverts commit 61e2ffcdefe964c8e74b74d8c10906ae29f32b6d.

* Add back checkpoint conversion test

Signed-off-by: Virginia Adams <[email protected]>

* Setting requires grad to True everywhere

Signed-off-by: Virginia Adams <[email protected]>

* Updated config comments and simplified param group code

Signed-off-by: Virginia Adams <[email protected]>

* Added comment on frozen_model having lr=0.0

Signed-off-by: Virginia Adams <[email protected]>

* Added configure optimizers methods

Signed-off-by: Virginia Adams <[email protected]>

* Set amp_o2 to false

Signed-off-by: Virginia Adams <[email protected]>

* removed o2 code

Signed-off-by: Virginia Adams <[email protected]>

* Python formatting fix

Signed-off-by: Virginia Adams <[email protected]>

Co-authored-by: Eric Harper <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* auto switch conformer encoder adapter in_features (#4354)

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Dataloader, collector, loss and metric for multiscale diarization decoder  (#4187)

* First commit

Signed-off-by: Taejin Park <[email protected]>

* Checked funtionality and imports

Signed-off-by: Taejin Park <[email protected]>

* fixed import issues

Signed-off-by: Taejin Park <[email protected]>

* Removed the changed made by mistake

Signed-off-by: Taejin Park <[email protected]>

* Style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM errors 001

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM and style fix

Signed-off-by: Taejin Park <[email protected]>

* Changed docstrings

Signed-off-by: Taejin Park <[email protected]>

* LGTM again

Signed-off-by: Taejin Park <[email protected]>

* Removed unnecessary torch setting lines

Signed-off-by: Taejin Park <[email protected]>

* Style fix and isort

Signed-off-by: Taejin Park <[email protected]>

* jbalam-nv comments reflected

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected comments and created _diar_label.py

Signed-off-by: Taejin Park <[email protected]>

* Typo fix and style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed target_spks[0] index error

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* LGTM unused import IterDataset

Signed-off-by: Taejin Park <[email protected]>

* revert collection doc year

Signed-off-by: Taejin Park <[email protected]>

* Code format error in collections.py

Signed-off-by: Taejin Park <[email protected]>

* fix collections space format error

Signed-off-by: Taejin Park <[email protected]>

* merged main correctly

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected all comments and tested

Signed-off-by: Taejin Park <[email protected]>

* style fix and LGTM

Signed-off-by: Taejin Park <[email protected]>

* rttm_filepath to rttm_file and removed self included funcs, tested

Signed-off-by: Taejin Park <[email protected]>

Co-authored-by: Nithin Rao <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Add ASR CTC Decoding module (#4342)

* Initial commit

Signed-off-by: smajumdar <[email protected]>

* Full support for decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Temp

Signed-off-by: smajumdar <[email protected]>

* Fix labels of y_sequence

Signed-off-by: smajumdar <[email protected]>

* Set support for sentencepiece subword merging

Signed-off-by: smajumdar <[email protected]>

* Fix char and word based token merge alignment

Signed-off-by: smajumdar <[email protected]>

* Revert incorrect change

Signed-off-by: smajumdar <[email protected]>

* Update docstring

Signed-off-by: smajumdar <[email protected]>

* Improve compatibility with greedy tokens and log probs

Signed-off-by: smajumdar <[email protected]>

* Update scripts to use decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix decoding of ctc models

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Option to disable mp in VAD via num_workers=1 (#4317)

* Option to disable mp in VAD via num_workers=1

In certain environments python multiprocessing can deadlock. This adds a convenient version to disable by setting num_workers to 1.

Signed-off-by: Georg Kucsko <[email protected]>

* add none handling

Signed-off-by: Georg Kucsko <[email protected]>

* additional none handling

Signed-off-by: Georg Kucsko <[email protected]>

Co-authored-by: fayejf <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* remove redundant bias expand (#4382)

* remove redundant bias expand

Signed-off-by: Xiaowei Ren <[email protected]>

* delete redundant code

Signed-off-by: Xiaowei Ren <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Add option for specifying wandb save_dir from config (#4379)

* give option to user to specify wandb save dir via config

Signed-off-by: Shantanu Acharya <[email protected]>

* create save_dir directory for wandb logger if not exists

Signed-off-by: Shantanu Acharya <[email protected]>

* update save_dir get method with a default value

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* [Bugfix][TTS] wrong order of returned tuple for general_collate_fn. (#4388)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Merge r1.10.0 main (#4398)

* update branch

Signed-off-by: ericharper <[email protected]>

* Set headscale false (#4364)

Signed-off-by: MaximumEntropy <[email protected]>

* Add wandb as dependency (#4365)

Signed-off-by: smajumdar <[email protected]>

* Raise trainer error (#4356)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Set headscale false (#4364) (#4366)

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: smajumdar <[email protected]>

* Finetuning changes for BART (#4003)

* Temp

Signed-off-by: MaximumEntropy <[email protected]>

* Checkpoint converter to nemo for bart

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Make position embedding expansion specific to a batch to avoid checkpoint size mismatches (#4357)

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Fix logging warning

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Fix electronic bug, new time ITN rule (#4355)

* fix electronic bug

Signed-off-by: ekmb <[email protected]>

* add new itn time rule

Signed-off-by: ekmb <[email protected]>

* revert domain changes

Signed-off-by: ekmb <[email protected]>

* remove repetition

Signed-off-by: ekmb <[email protected]>

* Correct support for dataclasses in default module dim (#4372)

* Correct support for dataclasses in default module dim

Signed-off-by: smajumdar <[email protected]>

* Fix path for save of results

Signed-off-by: smajumdar <[email protected]>

* fix pad id bug (#4377)

Signed-off-by: Yi Dong <[email protected]>

* Question answering bug fix (#4381)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assistant data processsor

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins for SGDGEN local checkpoint

Signed-off-by: Zhilin Wang <[email protected]>

* update style

Signed-off-by: Zhilin Wang <[email protected]>

* use local vocab file for Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* patch for Jenkins CI using local file

Signed-off-by: Zhilin Wang <[email protected]>

* add slot filling prediction and metrics

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused code

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

…
piraka9011 added a commit to piraka9011/NeMo that referenced this pull request Aug 25, 2022
* stateless RNNT working

Signed-off-by: Hainan Xu <[email protected]>

* batch decode working

Signed-off-by: Hainan Xu <[email protected]>

* working backup

Signed-off-by: Hainan Xu <[email protected]>

* good working version

Signed-off-by: Hainan Xu <[email protected]>

* temporarily make norm layer have affine

Signed-off-by: Hainan Xu <[email protected]>

* temp

Signed-off-by: Hainan Xu <[email protected]>

* temp

Signed-off-by: Hainan Xu <[email protected]>

* [TTS] add staticmethod decoration for BetaBinomialInterpolator (#4319)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* [TTS] remove redundant lines and declare global variables and capture (#4320)

exception of non-supported windows.

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Finetune T5 on the prefix-lm objective (#4328)

* Add script and yaml config

Signed-off-by: MaximumEntropy <[email protected]>

* Fix yaml config

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Update yaml to remove hardcoded model path

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Oleksii Kuchaiev <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Fuse bias with geglu in ParallelMLP (#4213)

* add code of fused_bias_geglu

* call fused_bias_geglu in ParallelMLP

* fix some bugs

* change biad_gelu_activation to bias_activation_fusion

* fix the setting of bias_actication_fusion for T5

* delete bias_gelu_fusion from T5 example config

* push reformatted files

* hto4h gemms fusion

* remove hto4h gemms fusion

* push reformatted files

* disable bias_activation_fusion while activation is not geglu

* add bias_activation_fusion in yaml config file

* add bias_gelu_fusion in T5 config yaml file to pass CI test

* change bias_gelu_fusion to bias_activation_fusion for T5 CI test

* recover latest change

Co-authored-by: Sandeep Subramanian <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Support larger datasets for question answering  (#4205)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assistant data processsor

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins for SGDGEN local checkpoint

Signed-off-by: Zhilin Wang <[email protected]>

* update style

Signed-off-by: Zhilin Wang <[email protected]>

* use local vocab file for Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* patch for Jenkins CI using local file

Signed-off-by: Zhilin Wang <[email protected]>

* add slot filling prediction and metrics

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused code

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* refactor metrics code out of Dialogue GPT Model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate backward compatible support for IntentSlotClassificationModel (bert model)

Signed-off-by: Zhilin Wang <[email protected]>

* save prediction file for IntentSlotClassification

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue gpt model training for megatron gpt

Signed-off-by: Zhilin Wang <[email protected]>

* remove batch generate for HF GPT2, which causes lower performance

Signed-off-by: Zhilin Wang <[email protected]>

* add few shot capability to dialogue gpt model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile and remove unused import

Signed-off-by: Zhilin Wang <[email protected]>

* update code description and clarity

Signed-off-by: Zhilin Wang <[email protected]>

* address PR comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate compatibility with ZeroShotIntentModel

Signed-off-by: Zhilin Wang <[email protected]>

* rename folder to dialogue due to increased scope and further refactor for clarity

Signed-off-by: Zhilin Wang <[email protected]>

* added dialogue GPT for sequence generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* add CI test for DialogueGPTGenerationModel

Signed-off-by: Zhilin Wang <[email protected]>

* integrate DialogueS2SGenerationModel for generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* modify huggingface utils to support HF t5/BART models

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* fix bleu metric style

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* update based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 2 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 3 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* integrate sgd generation based on user user utterance and system slot-values to generate system utterance

Signed-off-by: Zhilin Wang <[email protected]>

* add validation model saving capabilities

Signed-off-by: Zhilin Wang <[email protected]>

* cleaned up code for SGD Based Answer extender

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue Generation CI

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* fix Jenkins CI issue"

Signed-off-by: Zhilin Wang <[email protected]>

* add support for design dataset

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary imports

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support megatron for dialogue_s2s_generation_model

Signed-off-by: Zhilin Wang <[email protected]>

* reduce loaded samples in MSMarcoDataProcessor to 64 when cfg.model.dataset.debug_mode=True

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update CI

Signed-off-by: Zhilin Wang <[email protected]>

* update checkpoint and predictions filename to include epoch number

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate HF BART MNLI into zero shot intent model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Nearest Neighbour Model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* refactor Dialogue SGD Data Processor to make interface for models cleaner

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue S2S Generation model for DialogueSGDDataProcessor interface

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support sgd and drive thru datasets by zero shot model and nearest neighbour model

Signed-off-by: Zhilin Wang <[email protected]>

* add prediction saving code to nearest neighbour and zero shot intent models

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo in sgd data processor

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Mellon QA Data Processor

Signed-off-by: Zhilin Wang <[email protected]>

* update mellon qa

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue.py to remove outdated info

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* add dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* address review comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix for cfg

Signed-off-by: Zhilin Wang <[email protected]>

* make dependency on apex optional

Signed-off-by: Zhilin Wang <[email protected]>

* change NLPDDPluggin calling logic to make it possible to run without apex

Signed-off-by: Zhilin Wang <[email protected]>

* add first draft of tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* reduce ms marco size by removing lines without wellFormedAnswers

Signed-off-by: Zhilin Wang <[email protected]>

* address pr comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update colab tutorial link in dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* include unit test and some refactor to facilitate unit test

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* address pr issues

Signed-off-by: Zhilin Wang <[email protected]>

* remove typos in dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* support larger files for question answering

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary artifacts to reduce memory use

Signed-off-by: Zhilin Wang <[email protected]>

* put 0 tensor to device

Signed-off-by: Zhilin Wang <[email protected]>

* update link within dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously delete files

Signed-off-by: Zhilin Wang <[email protected]>

* update error handling when loss = nan

Signed-off-by: Zhilin Wang <[email protected]>

* update nan handling

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss func

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss

Signed-off-by: Zhilin Wang <[email protected]>

* fix type error raised in qa_dataset.py

Signed-off-by: Zhilin Wang <[email protected]>

* add error checking message

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update exp logging

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* limit number of negative samples

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused methods and style fix

Signed-off-by: Zhilin Wang <[email protected]>

* add more documentation

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* changes base on PR review

Signed-off-by: Zhilin Wang <[email protected]>

Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Fix bugs in indexed dataset exam script (#4325)

* fix the typo

Signed-off-by: Yi Dong <[email protected]>

* add neighbors option

Signed-off-by: Yi Dong <[email protected]>

* change the argument name

Signed-off-by: Yi Dong <[email protected]>

Co-authored-by: Micha Livne <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Adding docs for ASR SSL (#4303)

* Initial commit for SSL docs

Signed-off-by: Krishna Puvvada <[email protected]>

* ssl docs update-1

Signed-off-by: Krishna Puvvada <[email protected]>

* ssl docs update-2

Signed-off-by: Krishna Puvvada <[email protected]>

Co-authored-by: Krishna Puvvada <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Fuse grad division into async grad allreduce (#4327)

* O2 runs but O1 does not

Signed-off-by: ericharper <[email protected]>

* disable async for O1

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

* update async flag in configure_optimizers

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

* revert

Signed-off-by: ericharper <[email protected]>

* update _require if using async

Signed-off-by: ericharper <[email protected]>

* clean comments

Signed-off-by: ericharper <[email protected]>

* always all_reduce

Signed-off-by: ericharper <[email protected]>

* add async grad allreduce and chunk optimization to T5

* push reformatted files after style check

* set chunk size as 0 while async grad allreduce is off

* more experiments show that 125MB is a better default chunk size for most cases

* add grad_allreduce_chunk_size_mb for GPT-3

* at the end of each training step, wait until all async grad allreduce works are done

* replace individual allreduce work.wait() with a single dGPU evice synchroonization

* add code of fused_bias_geglu

* call fused_bias_geglu in ParallelMLP

* record the status of each allreduce work seems too much for perf

* add more comments

* push a reformatted file

* fix some bugs

* change biad_gelu_activation to bias_activation_fusion

* fix the setting of bias_actication_fusion for T5

* delete bias_gelu_fusion from T5 example config

* push reformatted files

* fuse grad scale with allreduce

* push reformatted files

* hto4h gemms fusion

* remove hto4h gemms fusion

* add grad_scale_ar_fusion into GPT-3

* push reformatted files

* push reformatted files

* rename grad_scale_ar_fusion to grad_div_ar_fusion

* disable bias_activation_fusion while activation is not geglu

* add bias_activation_fusion in yaml config file

* add bias_gelu_fusion in T5 config yaml file to pass CI test

* change bias_gelu_fusion to bias_activation_fusion for T5 CI test

* recover latest change

* add grad_div_ar_fusion in config yaml file

* remove a redundant float()

Co-authored-by: ericharper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Update container to 22.05 (#4329)

* update container to 22.05

Signed-off-by: ericharper <[email protected]>

* try adding safe directory

Signed-off-by: ericharper <[email protected]>

* try env var

Signed-off-by: ericharper <[email protected]>

* printenv

Signed-off-by: ericharper <[email protected]>

* try GIT_BRANCH

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

* remove dbug statements

Signed-off-by: ericharper <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Torchaudio installation fix (#4330)

* separate installer added

Signed-off-by: Aleksandr Laptev <[email protected]>

* apply suggestions, minor fixes

Signed-off-by: Aleksandr Laptev <[email protected]>

Co-authored-by: Aleksandr Laptev <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* [TTS] enforced pin_memory = True (#4341)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Merge r1.9.0 main (#4331)

* update branch

Signed-off-by: ericharper <[email protected]>

* update package info

Signed-off-by: ericharper <[email protected]>

* cleaned up TN/ ITN doc (#4119)

* cleaned up TN/ ITN doc

Signed-off-by: Yang Zhang <[email protected]>

* fix typo

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* Draft: Fix restoring from checkpoint for case when `model.common_dataset_parameters.label_vocab_dir` is provided (#4136)

* Fix restoring from checkpoint with label vocab dir

Signed-off-by: PeganovAnton <[email protected]>

* Add tests for various ways to pass label ids to model

Signed-off-by: PeganovAnton <[email protected]>

* Fix typo

Signed-off-by: PeganovAnton <[email protected]>

* Fix typo

Signed-off-by: PeganovAnton <[email protected]>

* Do not create tmp directory

Signed-off-by: PeganovAnton <[email protected]>

* Fix parameter name

Signed-off-by: PeganovAnton <[email protected]>

* finish cherry-pick op

Signed-off-by: PeganovAnton <[email protected]>

* Fix labels errors

Signed-off-by: PeganovAnton <[email protected]>

* Remove duplicate stage

Signed-off-by: PeganovAnton <[email protected]>

* Change target branch

Signed-off-by: PeganovAnton <[email protected]>

* fix doc (#4146)

Signed-off-by: Yang Zhang <[email protected]>

* Tacotron2 retrain (#4103)

* fix yaml

Signed-off-by: treacker <[email protected]>

* Fix for new TTSDataset class

Signed-off-by: treacker <[email protected]>

* added wandb logging

Signed-off-by: treacker <[email protected]>

* added wandb logging

Signed-off-by: treacker <[email protected]>

* fix numpy version

Signed-off-by: treacker <[email protected]>

* fix numpy version

Signed-off-by: treacker <[email protected]>

* inference fix

Signed-off-by: treacker <[email protected]>

* removed old code

Signed-off-by: treacker <[email protected]>

* updated parser logic

Signed-off-by: treacker <[email protected]>

* reverted version update

Signed-off-by: treacker <[email protected]>

* refactored parser logic

Signed-off-by: treacker <[email protected]>

* Updated Jenkinsfile

Signed-off-by: treacker <[email protected]>

* Refactored tutorial for Tacotron2

Signed-off-by: treacker <[email protected]>

* Made backward compatibility

Signed-off-by: treacker <[email protected]>

* Made backward compatibility

Signed-off-by: treacker <[email protected]>

* Update Jenkinsfile

Signed-off-by: treacker <[email protected]>

* Update tacotron.yaml

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* cleaned up TN/ ITN doc (#4119)

* cleaned up TN/ ITN doc

Signed-off-by: Yang Zhang <[email protected]>

* fix typo

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>
Signed-off-by: treacker <[email protected]>

* Check implicit grad acc in GLUE dataset building (#4123)

* Check implicit grad acc in GLUE dataset building

Signed-off-by: MaximumEntropy <[email protected]>

* Fix jenkins test for GLUE/XNLI

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Fixed jenkins

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>

* Multiprocess improvements (#4127)

* initial commit

Signed-off-by: nithinraok <[email protected]>

* start fix

Signed-off-by: nithinraok <[email protected]>

* improve multiprocessing speed while creating speaker dataset

Signed-off-by: nithinraok <[email protected]>

* updated scp to filelist

Signed-off-by: nithinraok <[email protected]>

* notebooks' link, typo and import  fix  (#4158)

* redo missing pr 4007

Signed-off-by: fayejf <[email protected]>

* remove extremely unreliable links

Signed-off-by: fayejf <[email protected]>

* update speaker docs (#4164)

* update speaker docs

Signed-off-by: nithinraok <[email protected]>

* chunks -> segments

Signed-off-by: nithinraok <[email protected]>

* Khz -> kHz

Signed-off-by: nithinraok <[email protected]>

* small fix (#4180)

Signed-off-by: fayejf <[email protected]>

* fix the server key value problem (#4196)

Signed-off-by: Yi Dong <[email protected]>

* Fix/punctuation/trainer required for setting test data (#4199)

* Draft of fix

Signed-off-by: PeganovAnton <[email protected]>

* Add warnings and replace globa_step with current_epoch

Signed-off-by: PeganovAnton <[email protected]>

* Small improvements to warnings

Signed-off-by: PeganovAnton <[email protected]>

* Error and warning messages improvements

Signed-off-by: PeganovAnton <[email protected]>

* Replace self.trainer with self._trainer

Signed-off-by: PeganovAnton <[email protected]>

* Update ContextNet version (#4207)

Signed-off-by: smajumdar <[email protected]>

* fix bugs for dialogue tutorial (#4211)

Signed-off-by: Zhilin Wang <[email protected]>

* Dialogue tutorial fix (#4214)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* Add docs for Thutmose Tagger (#4173)

* Add docs for Thutmose Tagger

Signed-off-by: Alexandra Antonova <[email protected]>

* add level in docs

Signed-off-by: Alexandra Antonova <[email protected]>

* delete folder to avoid error with running when folder exists from previous run

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>
Co-authored-by: ekmb <[email protected]>

* Dialogue tutorial fix (#4218)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously deleted files

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* Dialogue tutorial fix (#4221)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously deleted files

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* fix syntax error in ipynb-file (#4228)

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>

* fix json serialize (#4235)

Signed-off-by: Yi Dong <[email protected]>

* Prompt Learning Typo Fixes (#4238)

* Prompt tuning notebook typo fixes

Signed-off-by: Virginia Adams <[email protected]>

* Update tutorials.rst

* Update prompt_learning.rst

* Update prompt_learning.rst

* fixing bug 3642622 (#4250)

* fixing bug 3642622

Signed-off-by: Ghasem Pasandi <[email protected]>

* fixing bug 3642622

Signed-off-by: Ghasem Pasandi <[email protected]>

Co-authored-by: Ghasem Pasandi <[email protected]>

* fix broken link in the tutorial (#4257)

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>

* Typo fix, branch change, better download messagae (#4262)

Signed-off-by: Virginia Adams <[email protected]>

* Raise error if bicleaner is not installed in NMT Data preprocesing notebook (#4264)

* Raise error if bicleaner is not installed

Signed-off-by: MaximumEntropy <[email protected]>

* Clear cells

Signed-off-by: MaximumEntropy <[email protected]>

* Fix missing validation dataset, whitelist certain keywords for datasets (#4269)

* Fix missing validation dataset, whitelist certain keywords for datasets

Signed-off-by: smajumdar <[email protected]>

* Fix missing validation dataset, whitelist certain keywords for datasets

Signed-off-by: smajumdar <[email protected]>

* Update asr configs with num_workers and pin_memory (#4270)

Signed-off-by: smajumdar <[email protected]>

* Fix epoch end (#4265)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* Set Save on train end to false (#4274)

* Set Save on train end to false

Signed-off-by: Virginia Adams <[email protected]>

* Update prompt_learning.rst

* Update prompt_learning.rst

* Update YAML (#4261)

Signed-off-by: MaximumEntropy <[email protected]>

* Updated config to fix CI test OOM error (#4279)

* Updated config to fix CI test issue

Signed-off-by: Virginia Adams <[email protected]>

* Increased num workers

Signed-off-by: Virginia Adams <[email protected]>

* verbose k2 install, skip if failed (#4289)

Signed-off-by: Aleksandr Laptev <[email protected]>

Co-authored-by: Aleksandr Laptev <[email protected]>

* Changed total virtual prompt tokens (#4295)

* Changed total virtual prompt tokens

Signed-off-by: Virginia Adams <[email protected]>

* put number of workers back

Signed-off-by: Virginia Adams <[email protected]>

* upper bound lightning

Signed-off-by: ericharper <[email protected]>

* update branch

Signed-off-by: ericharper <[email protected]>

* update config

Signed-off-by: ericharper <[email protected]>

* remove duplicate test

Signed-off-by: ericharper <[email protected]>

* fix tn test cases

Signed-off-by: ericharper <[email protected]>

* add another safe.directory

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: PeganovAnton <[email protected]>
Co-authored-by: treacker <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Nithin Rao <[email protected]>
Co-authored-by: fayejf <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: bene-ges <[email protected]>
Co-authored-by: Alexandra Antonova <[email protected]>
Co-authored-by: ekmb <[email protected]>
Co-authored-by: Virginia Adams <[email protected]>
Co-authored-by: Ghasem <[email protected]>
Co-authored-by: Ghasem Pasandi <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* [TTS] Training Fastpitch on German text and phonemes and finetuning HiFi-GAN on predicted mels (#4266)

* initial commit

Signed-off-by: Akshit Arora <[email protected]>

* cleared notebook outputs

Signed-off-by: Akshit Arora <[email protected]>

* formatting errors

Signed-off-by: Akshit Arora <[email protected]>

* formatting

Signed-off-by: Akshit Arora <[email protected]>

* addressed comments

Signed-off-by: Akshit Arora <[email protected]>

* addressed comments on tutorial

Signed-off-by: Akshit Arora <[email protected]>

* updated tutorial

Signed-off-by: Akshit Arora <[email protected]>

* updated grammar and fastpitch description

Signed-off-by: Akshit Arora <[email protected]>

* updated with feedback

Signed-off-by: Akshit Arora <[email protected]>

* updated with feedback

Signed-off-by: Akshit Arora <[email protected]>

* removed unused imports

Signed-off-by: Akshit Arora <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Speedup the speech commands dataset processing script (#4347)

* Add multiprocessing support to the google speech commands dataset processing script

Signed-off-by: Shantanu Acharya <[email protected]>

* fix number of args error with __extract_all_files function

Signed-off-by: Shantanu Acharya <[email protected]>

* fix styling issues

Signed-off-by: Shantanu Acharya <[email protected]>

* fix bugs with silence set construction and update librosa output write to use soundfile write

Signed-off-by: Shantanu Acharya <[email protected]>

* add docstrings and return values in __construct_filepaths as dictionary

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* fix wrong requirement (#4349)

Signed-off-by: Yang Zhang <[email protected]>

Co-authored-by: Sandeep Subramanian <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Refactored path to manifest (#4251)

Signed-off-by: Evgeniy Shabalin <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* IPA support for TTS (#4310)

* IPA tokenizer and G2P untested draft

Signed-off-by: Jocelyn Huang <[email protected]>

* Add IPA CMUdict and new heteronyms list

Signed-off-by: Jocelyn Huang <[email protected]>

* Add draft FastPitch IPA config

Signed-off-by: Jocelyn Huang <[email protected]>

* Minor bugfixes for IPA training

Signed-off-by: Jocelyn Huang <[email protected]>

* Add phoneme_probability to IPA G2P

Signed-off-by: Jocelyn Huang <[email protected]>

* Updates to IPA FastPitch training config

Signed-off-by: Jocelyn Huang <[email protected]>

* Update IPA dict and heteronyms file

Signed-off-by: Jocelyn Huang <[email protected]>

* Adjust default lr for IPA FastPitch to 1e-3

Signed-off-by: Jocelyn Huang <[email protected]>

* Rename IPA CMUdict to reflect date

Signed-off-by: Jocelyn Huang <[email protected]>

* Add docstrings for IPA tokenizer and G2P, update CMUdict path for config

Signed-off-by: Jocelyn Huang <[email protected]>

* Fix IPA vocab ordering, add options to uppercase graphemes and remove stress symbols

Signed-off-by: Jocelyn Huang <[email protected]>

* Mark IPA classes as experimental

Signed-off-by: Jocelyn Huang <[email protected]>

* Update apostrophe-S cases

Signed-off-by: Jocelyn Huang <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Tn install (#4055)

* remove conda pynini requirement

Signed-off-by: Yang Zhang <[email protected]>

* remove remnants

Signed-off-by: Yang Zhang <[email protected]>

* merge with main

Signed-off-by: Yang Zhang <[email protected]>

* removing nlp collection dependency from text processing and thus breaking cyclyc imports

Signed-off-by: Yang Zhang <[email protected]>

* fix

Signed-off-by: Yang Zhang <[email protected]>

* fix wrong requirement

Signed-off-by: Yang Zhang <[email protected]>

* fix bug in vi

Signed-off-by: Yang Zhang <[email protected]>

* update jenkins folders

Signed-off-by: ekmb <[email protected]>

Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: ekmb <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* fix tutorial (#4352)

Signed-off-by: stevehuang52 <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* fix the post ln (#4350)

Signed-off-by: Yi Dong <[email protected]>

Co-authored-by: Eric Harper <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* [Fix] Hanging for Fully Randomized Bucketing (#4348)

* Update container to 22.05 (#4329)

* update container to 22.05

Signed-off-by: ericharper <[email protected]>

* try adding safe directory

Signed-off-by: ericharper <[email protected]>

* try env var

Signed-off-by: ericharper <[email protected]>

* printenv

Signed-off-by: ericharper <[email protected]>

* try GIT_BRANCH

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

* remove dbug statements

Signed-off-by: ericharper <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>

* Merge r1.9.0 main (#4331)

* update branch

Signed-off-by: ericharper <[email protected]>

* update package info

Signed-off-by: ericharper <[email protected]>

* cleaned up TN/ ITN doc (#4119)

* cleaned up TN/ ITN doc

Signed-off-by: Yang Zhang <[email protected]>

* fix typo

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* Draft: Fix restoring from checkpoint for case when `model.common_dataset_parameters.label_vocab_dir` is provided (#4136)

* Fix restoring from checkpoint with label vocab dir

Signed-off-by: PeganovAnton <[email protected]>

* Add tests for various ways to pass label ids to model

Signed-off-by: PeganovAnton <[email protected]>

* Fix typo

Signed-off-by: PeganovAnton <[email protected]>

* Fix typo

Signed-off-by: PeganovAnton <[email protected]>

* Do not create tmp directory

Signed-off-by: PeganovAnton <[email protected]>

* Fix parameter name

Signed-off-by: PeganovAnton <[email protected]>

* finish cherry-pick op

Signed-off-by: PeganovAnton <[email protected]>

* Fix labels errors

Signed-off-by: PeganovAnton <[email protected]>

* Remove duplicate stage

Signed-off-by: PeganovAnton <[email protected]>

* Change target branch

Signed-off-by: PeganovAnton <[email protected]>

* fix doc (#4146)

Signed-off-by: Yang Zhang <[email protected]>

* Tacotron2 retrain (#4103)

* fix yaml

Signed-off-by: treacker <[email protected]>

* Fix for new TTSDataset class

Signed-off-by: treacker <[email protected]>

* added wandb logging

Signed-off-by: treacker <[email protected]>

* added wandb logging

Signed-off-by: treacker <[email protected]>

* fix numpy version

Signed-off-by: treacker <[email protected]>

* fix numpy version

Signed-off-by: treacker <[email protected]>

* inference fix

Signed-off-by: treacker <[email protected]>

* removed old code

Signed-off-by: treacker <[email protected]>

* updated parser logic

Signed-off-by: treacker <[email protected]>

* reverted version update

Signed-off-by: treacker <[email protected]>

* refactored parser logic

Signed-off-by: treacker <[email protected]>

* Updated Jenkinsfile

Signed-off-by: treacker <[email protected]>

* Refactored tutorial for Tacotron2

Signed-off-by: treacker <[email protected]>

* Made backward compatibility

Signed-off-by: treacker <[email protected]>

* Made backward compatibility

Signed-off-by: treacker <[email protected]>

* Update Jenkinsfile

Signed-off-by: treacker <[email protected]>

* Update tacotron.yaml

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* cleaned up TN/ ITN doc (#4119)

* cleaned up TN/ ITN doc

Signed-off-by: Yang Zhang <[email protected]>

* fix typo

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>
Signed-off-by: treacker <[email protected]>

* Check implicit grad acc in GLUE dataset building (#4123)

* Check implicit grad acc in GLUE dataset building

Signed-off-by: MaximumEntropy <[email protected]>

* Fix jenkins test for GLUE/XNLI

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Fixed jenkins

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>

* Multiprocess improvements (#4127)

* initial commit

Signed-off-by: nithinraok <[email protected]>

* start fix

Signed-off-by: nithinraok <[email protected]>

* improve multiprocessing speed while creating speaker dataset

Signed-off-by: nithinraok <[email protected]>

* updated scp to filelist

Signed-off-by: nithinraok <[email protected]>

* notebooks' link, typo and import  fix  (#4158)

* redo missing pr 4007

Signed-off-by: fayejf <[email protected]>

* remove extremely unreliable links

Signed-off-by: fayejf <[email protected]>

* update speaker docs (#4164)

* update speaker docs

Signed-off-by: nithinraok <[email protected]>

* chunks -> segments

Signed-off-by: nithinraok <[email protected]>

* Khz -> kHz

Signed-off-by: nithinraok <[email protected]>

* small fix (#4180)

Signed-off-by: fayejf <[email protected]>

* fix the server key value problem (#4196)

Signed-off-by: Yi Dong <[email protected]>

* Fix/punctuation/trainer required for setting test data (#4199)

* Draft of fix

Signed-off-by: PeganovAnton <[email protected]>

* Add warnings and replace globa_step with current_epoch

Signed-off-by: PeganovAnton <[email protected]>

* Small improvements to warnings

Signed-off-by: PeganovAnton <[email protected]>

* Error and warning messages improvements

Signed-off-by: PeganovAnton <[email protected]>

* Replace self.trainer with self._trainer

Signed-off-by: PeganovAnton <[email protected]>

* Update ContextNet version (#4207)

Signed-off-by: smajumdar <[email protected]>

* fix bugs for dialogue tutorial (#4211)

Signed-off-by: Zhilin Wang <[email protected]>

* Dialogue tutorial fix (#4214)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* Add docs for Thutmose Tagger (#4173)

* Add docs for Thutmose Tagger

Signed-off-by: Alexandra Antonova <[email protected]>

* add level in docs

Signed-off-by: Alexandra Antonova <[email protected]>

* delete folder to avoid error with running when folder exists from previous run

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>
Co-authored-by: ekmb <[email protected]>

* Dialogue tutorial fix (#4218)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously deleted files

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* Dialogue tutorial fix (#4221)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously deleted files

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* fix syntax error in ipynb-file (#4228)

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>

* fix json serialize (#4235)

Signed-off-by: Yi Dong <[email protected]>

* Prompt Learning Typo Fixes (#4238)

* Prompt tuning notebook typo fixes

Signed-off-by: Virginia Adams <[email protected]>

* Update tutorials.rst

* Update prompt_learning.rst

* Update prompt_learning.rst

* fixing bug 3642622 (#4250)

* fixing bug 3642622

Signed-off-by: Ghasem Pasandi <[email protected]>

* fixing bug 3642622

Signed-off-by: Ghasem Pasandi <[email protected]>

Co-authored-by: Ghasem Pasandi <[email protected]>

* fix broken link in the tutorial (#4257)

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>

* Typo fix, branch change, better download messagae (#4262)

Signed-off-by: Virginia Adams <[email protected]>

* Raise error if bicleaner is not installed in NMT Data preprocesing notebook (#4264)

* Raise error if bicleaner is not installed

Signed-off-by: MaximumEntropy <[email protected]>

* Clear cells

Signed-off-by: MaximumEntropy <[email protected]>

* Fix missing validation dataset, whitelist certain keywords for datasets (#4269)

* Fix missing validation dataset, whitelist certain keywords for datasets

Signed-off-by: smajumdar <[email protected]>

* Fix missing validation dataset, whitelist certain keywords for datasets

Signed-off-by: smajumdar <[email protected]>

* Update asr configs with num_workers and pin_memory (#4270)

Signed-off-by: smajumdar <[email protected]>

* Fix epoch end (#4265)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* Set Save on train end to false (#4274)

* Set Save on train end to false

Signed-off-by: Virginia Adams <[email protected]>

* Update prompt_learning.rst

* Update prompt_learning.rst

* Update YAML (#4261)

Signed-off-by: MaximumEntropy <[email protected]>

* Updated config to fix CI test OOM error (#4279)

* Updated config to fix CI test issue

Signed-off-by: Virginia Adams <[email protected]>

* Increased num workers

Signed-off-by: Virginia Adams <[email protected]>

* verbose k2 install, skip if failed (#4289)

Signed-off-by: Aleksandr Laptev <[email protected]>

Co-authored-by: Aleksandr Laptev <[email protected]>

* Changed total virtual prompt tokens (#4295)

* Changed total virtual prompt tokens

Signed-off-by: Virginia Adams <[email protected]>

* put number of workers back

Signed-off-by: Virginia Adams <[email protected]>

* upper bound lightning

Signed-off-by: ericharper <[email protected]>

* update branch

Signed-off-by: ericharper <[email protected]>

* update config

Signed-off-by: ericharper <[email protected]>

* remove duplicate test

Signed-off-by: ericharper <[email protected]>

* fix tn test cases

Signed-off-by: ericharper <[email protected]>

* add another safe.directory

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: PeganovAnton <[email protected]>
Co-authored-by: treacker <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Nithin Rao <[email protected]>
Co-authored-by: fayejf <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: bene-ges <[email protected]>
Co-authored-by: Alexandra Antonova <[email protected]>
Co-authored-by: ekmb <[email protected]>
Co-authored-by: Virginia Adams <[email protected]>
Co-authored-by: Ghasem <[email protected]>
Co-authored-by: Ghasem Pasandi <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>

* fix full_randn bucket hang

Signed-off-by: stevehuang52 <[email protected]>

* remove unused variables

Signed-off-by: stevehuang52 <[email protected]>

Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: PeganovAnton <[email protected]>
Co-authored-by: treacker <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Nithin Rao <[email protected]>
Co-authored-by: fayejf <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: bene-ges <[email protected]>
Co-authored-by: Alexandra Antonova <[email protected]>
Co-authored-by: ekmb <[email protected]>
Co-authored-by: Virginia Adams <[email protected]>
Co-authored-by: Ghasem <[email protected]>
Co-authored-by: Ghasem Pasandi <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Bits of RADTTS support (#4343)

* Bits of RADTTS support

Signed-off-by: Boris Fomitchev <[email protected]>

* Fixed args mismatch

Signed-off-by: Boris Fomitchev <[email protected]>

* Style

Signed-off-by: Boris Fomitchev <[email protected]>

* Addressed review comments

Signed-off-by: Boris Fomitchev <[email protected]>

* More review comments

Signed-off-by: Boris Fomitchev <[email protected]>

Co-authored-by: Eric Harper <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Prompt Learning Pipeline Parallel (#4291)

* Added get_forward_output_and_loss_func and updated train/val steps

Signed-off-by: Virginia Adams <[email protected]>

* Added preprocess flag before prompt table/encoder access

Signed-off-by: Virginia Adams <[email protected]>

* Made two optimizer groups, one for frozen, one for soft prompt

Signed-off-by: Virginia Adams <[email protected]>

* Pipeline parallel working

Signed-off-by: Virginia Adams <[email protected]>

* Still figuring out setting lr/sched for one param group

Signed-off-by: Virginia Adams <[email protected]>

* Set betas to zero

Signed-off-by: Virginia Adams <[email protected]>

* Only unfreeze one sublayer with lr 0.0

Signed-off-by: Virginia Adams <[email protected]>

* Pipeline parallel working w/ one optimizer

Signed-off-by: Virginia Adams <[email protected]>

* Trying to fix Jenkins file

Signed-off-by: Virginia Adams <[email protected]>

* Trying to fix Jenkins file

Signed-off-by: Virginia Adams <[email protected]>

* Getting updated jenkins test to work

Signed-off-by: Virginia Adams <[email protected]>

* Getting updated jenkins test to work

Signed-off-by: Virginia Adams <[email protected]>

* added prompt learning tp and pp CI tests

Signed-off-by: Virginia Adams <[email protected]>

* Added amp_o2 model compatibility

Signed-off-by: Virginia Adams <[email protected]>

* Made CI test smaller

Signed-off-by: Virginia Adams <[email protected]>

* Still trying to get Jenkins to work

Signed-off-by: Virginia Adams <[email protected]>

* Still trying to get Jenkins to work

Signed-off-by: Virginia Adams <[email protected]>

* Temporarily moving prompt learning CI test to beginning

Signed-off-by: Virginia Adams <[email protected]>

* Changing the layer being unfrozen

Signed-off-by: Virginia Adams <[email protected]>

* debug jenkins

Signed-off-by: Virginia Adams <[email protected]>

* Move pp unfreeze to init

Signed-off-by: Virginia Adams <[email protected]>

* Try to make Jenkins test parallel

Signed-off-by: Virginia Adams <[email protected]>

* Fix python formatting

Signed-off-by: Virginia Adams <[email protected]>

* Moved prompt learning tests back to where they belong

Signed-off-by: Virginia Adams <[email protected]>

* add back checkpoint convertion CI test

Signed-off-by: Virginia Adams <[email protected]>

* Revert "add back checkpoint convertion CI test"

This reverts commit 61e2ffcdefe964c8e74b74d8c10906ae29f32b6d.

* Add back checkpoint conversion test

Signed-off-by: Virginia Adams <[email protected]>

* Setting requires grad to True everywhere

Signed-off-by: Virginia Adams <[email protected]>

* Updated config comments and simplified param group code

Signed-off-by: Virginia Adams <[email protected]>

* Added comment on frozen_model having lr=0.0

Signed-off-by: Virginia Adams <[email protected]>

* Added configure optimizers methods

Signed-off-by: Virginia Adams <[email protected]>

* Set amp_o2 to false

Signed-off-by: Virginia Adams <[email protected]>

* removed o2 code

Signed-off-by: Virginia Adams <[email protected]>

* Python formatting fix

Signed-off-by: Virginia Adams <[email protected]>

Co-authored-by: Eric Harper <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* auto switch conformer encoder adapter in_features (#4354)

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Dataloader, collector, loss and metric for multiscale diarization decoder  (#4187)

* First commit

Signed-off-by: Taejin Park <[email protected]>

* Checked funtionality and imports

Signed-off-by: Taejin Park <[email protected]>

* fixed import issues

Signed-off-by: Taejin Park <[email protected]>

* Removed the changed made by mistake

Signed-off-by: Taejin Park <[email protected]>

* Style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM errors 001

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM and style fix

Signed-off-by: Taejin Park <[email protected]>

* Changed docstrings

Signed-off-by: Taejin Park <[email protected]>

* LGTM again

Signed-off-by: Taejin Park <[email protected]>

* Removed unnecessary torch setting lines

Signed-off-by: Taejin Park <[email protected]>

* Style fix and isort

Signed-off-by: Taejin Park <[email protected]>

* jbalam-nv comments reflected

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected comments and created _diar_label.py

Signed-off-by: Taejin Park <[email protected]>

* Typo fix and style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed target_spks[0] index error

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* LGTM unused import IterDataset

Signed-off-by: Taejin Park <[email protected]>

* revert collection doc year

Signed-off-by: Taejin Park <[email protected]>

* Code format error in collections.py

Signed-off-by: Taejin Park <[email protected]>

* fix collections space format error

Signed-off-by: Taejin Park <[email protected]>

* merged main correctly

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected all comments and tested

Signed-off-by: Taejin Park <[email protected]>

* style fix and LGTM

Signed-off-by: Taejin Park <[email protected]>

* rttm_filepath to rttm_file and removed self included funcs, tested

Signed-off-by: Taejin Park <[email protected]>

Co-authored-by: Nithin Rao <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Add ASR CTC Decoding module (#4342)

* Initial commit

Signed-off-by: smajumdar <[email protected]>

* Full support for decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Temp

Signed-off-by: smajumdar <[email protected]>

* Fix labels of y_sequence

Signed-off-by: smajumdar <[email protected]>

* Set support for sentencepiece subword merging

Signed-off-by: smajumdar <[email protected]>

* Fix char and word based token merge alignment

Signed-off-by: smajumdar <[email protected]>

* Revert incorrect change

Signed-off-by: smajumdar <[email protected]>

* Update docstring

Signed-off-by: smajumdar <[email protected]>

* Improve compatibility with greedy tokens and log probs

Signed-off-by: smajumdar <[email protected]>

* Update scripts to use decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix decoding of ctc models

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Option to disable mp in VAD via num_workers=1 (#4317)

* Option to disable mp in VAD via num_workers=1

In certain environments python multiprocessing can deadlock. This adds a convenient version to disable by setting num_workers to 1.

Signed-off-by: Georg Kucsko <[email protected]>

* add none handling

Signed-off-by: Georg Kucsko <[email protected]>

* additional none handling

Signed-off-by: Georg Kucsko <[email protected]>

Co-authored-by: fayejf <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* remove redundant bias expand (#4382)

* remove redundant bias expand

Signed-off-by: Xiaowei Ren <[email protected]>

* delete redundant code

Signed-off-by: Xiaowei Ren <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Add option for specifying wandb save_dir from config (#4379)

* give option to user to specify wandb save dir via config

Signed-off-by: Shantanu Acharya <[email protected]>

* create save_dir directory for wandb logger if not exists

Signed-off-by: Shantanu Acharya <[email protected]>

* update save_dir get method with a default value

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* [Bugfix][TTS] wrong order of returned tuple for general_collate_fn. (#4388)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Merge r1.10.0 main (#4398)

* update branch

Signed-off-by: ericharper <[email protected]>

* Set headscale false (#4364)

Signed-off-by: MaximumEntropy <[email protected]>

* Add wandb as dependency (#4365)

Signed-off-by: smajumdar <[email protected]>

* Raise trainer error (#4356)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Set headscale false (#4364) (#4366)

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: smajumdar <[email protected]>

* Finetuning changes for BART (#4003)

* Temp

Signed-off-by: MaximumEntropy <[email protected]>

* Checkpoint converter to nemo for bart

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Make position embedding expansion specific to a batch to avoid checkpoint size mismatches (#4357)

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Fix logging warning

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Fix electronic bug, new time ITN rule (#4355)

* fix electronic bug

Signed-off-by: ekmb <[email protected]>

* add new itn time rule

Signed-off-by: ekmb <[email protected]>

* revert domain changes

Signed-off-by: ekmb <[email protected]>

* remove repetition

Signed-off-by: ekmb <[email protected]>

* Correct support for dataclasses in default module dim (#4372)

* Correct support for dataclasses in default module dim

Signed-off-by: smajumdar <[email protected]>

* Fix path for save of results

Signed-off-by: smajumdar <[email protected]>

* fix pad id bug (#4377)

Signed-off-by: Yi Dong <[email protected]>

* Question answering bug fix (#4381)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assistant data processsor

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins for SGDGEN local checkpoint

Signed-off-by: Zhilin Wang <[email protected]>

* update style

Signed-off-by: Zhilin Wang <[email protected]>

* use local vocab file for Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* patch for Jenkins CI using local file

Signed-off-by: Zhilin Wang <[email protected]>

* add slot filling prediction and metrics

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused code

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

…
hainan-xv added a commit to hainan-xv/NeMo that referenced this pull request Nov 29, 2022
* stateless RNNT working

Signed-off-by: Hainan Xu <[email protected]>

* batch decode working

Signed-off-by: Hainan Xu <[email protected]>

* working backup

Signed-off-by: Hainan Xu <[email protected]>

* good working version

Signed-off-by: Hainan Xu <[email protected]>

* temporarily make norm layer have affine

Signed-off-by: Hainan Xu <[email protected]>

* temp

Signed-off-by: Hainan Xu <[email protected]>

* temp

Signed-off-by: Hainan Xu <[email protected]>

* [TTS] add staticmethod decoration for BetaBinomialInterpolator (#4319)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* [TTS] remove redundant lines and declare global variables and capture (#4320)

exception of non-supported windows.

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Finetune T5 on the prefix-lm objective (#4328)

* Add script and yaml config

Signed-off-by: MaximumEntropy <[email protected]>

* Fix yaml config

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Update yaml to remove hardcoded model path

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Oleksii Kuchaiev <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Fuse bias with geglu in ParallelMLP (#4213)

* add code of fused_bias_geglu

* call fused_bias_geglu in ParallelMLP

* fix some bugs

* change biad_gelu_activation to bias_activation_fusion

* fix the setting of bias_actication_fusion for T5

* delete bias_gelu_fusion from T5 example config

* push reformatted files

* hto4h gemms fusion

* remove hto4h gemms fusion

* push reformatted files

* disable bias_activation_fusion while activation is not geglu

* add bias_activation_fusion in yaml config file

* add bias_gelu_fusion in T5 config yaml file to pass CI test

* change bias_gelu_fusion to bias_activation_fusion for T5 CI test

* recover latest change

Co-authored-by: Sandeep Subramanian <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Support larger datasets for question answering  (#4205)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assistant data processsor

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins for SGDGEN local checkpoint

Signed-off-by: Zhilin Wang <[email protected]>

* update style

Signed-off-by: Zhilin Wang <[email protected]>

* use local vocab file for Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* patch for Jenkins CI using local file

Signed-off-by: Zhilin Wang <[email protected]>

* add slot filling prediction and metrics

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused code

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* refactor metrics code out of Dialogue GPT Model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate backward compatible support for IntentSlotClassificationModel (bert model)

Signed-off-by: Zhilin Wang <[email protected]>

* save prediction file for IntentSlotClassification

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue gpt model training for megatron gpt

Signed-off-by: Zhilin Wang <[email protected]>

* remove batch generate for HF GPT2, which causes lower performance

Signed-off-by: Zhilin Wang <[email protected]>

* add few shot capability to dialogue gpt model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile and remove unused import

Signed-off-by: Zhilin Wang <[email protected]>

* update code description and clarity

Signed-off-by: Zhilin Wang <[email protected]>

* address PR comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate compatibility with ZeroShotIntentModel

Signed-off-by: Zhilin Wang <[email protected]>

* rename folder to dialogue due to increased scope and further refactor for clarity

Signed-off-by: Zhilin Wang <[email protected]>

* added dialogue GPT for sequence generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* add CI test for DialogueGPTGenerationModel

Signed-off-by: Zhilin Wang <[email protected]>

* integrate DialogueS2SGenerationModel for generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* modify huggingface utils to support HF t5/BART models

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* fix bleu metric style

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* update based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 2 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 3 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* integrate sgd generation based on user user utterance and system slot-values to generate system utterance

Signed-off-by: Zhilin Wang <[email protected]>

* add validation model saving capabilities

Signed-off-by: Zhilin Wang <[email protected]>

* cleaned up code for SGD Based Answer extender

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue Generation CI

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* fix Jenkins CI issue"

Signed-off-by: Zhilin Wang <[email protected]>

* add support for design dataset

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary imports

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support megatron for dialogue_s2s_generation_model

Signed-off-by: Zhilin Wang <[email protected]>

* reduce loaded samples in MSMarcoDataProcessor to 64 when cfg.model.dataset.debug_mode=True

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update CI

Signed-off-by: Zhilin Wang <[email protected]>

* update checkpoint and predictions filename to include epoch number

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate HF BART MNLI into zero shot intent model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Nearest Neighbour Model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* refactor Dialogue SGD Data Processor to make interface for models cleaner

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue S2S Generation model for DialogueSGDDataProcessor interface

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support sgd and drive thru datasets by zero shot model and nearest neighbour model

Signed-off-by: Zhilin Wang <[email protected]>

* add prediction saving code to nearest neighbour and zero shot intent models

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo in sgd data processor

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Mellon QA Data Processor

Signed-off-by: Zhilin Wang <[email protected]>

* update mellon qa

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue.py to remove outdated info

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* add dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* address review comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix for cfg

Signed-off-by: Zhilin Wang <[email protected]>

* make dependency on apex optional

Signed-off-by: Zhilin Wang <[email protected]>

* change NLPDDPluggin calling logic to make it possible to run without apex

Signed-off-by: Zhilin Wang <[email protected]>

* add first draft of tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* reduce ms marco size by removing lines without wellFormedAnswers

Signed-off-by: Zhilin Wang <[email protected]>

* address pr comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update colab tutorial link in dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* include unit test and some refactor to facilitate unit test

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* address pr issues

Signed-off-by: Zhilin Wang <[email protected]>

* remove typos in dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* support larger files for question answering

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary artifacts to reduce memory use

Signed-off-by: Zhilin Wang <[email protected]>

* put 0 tensor to device

Signed-off-by: Zhilin Wang <[email protected]>

* update link within dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously delete files

Signed-off-by: Zhilin Wang <[email protected]>

* update error handling when loss = nan

Signed-off-by: Zhilin Wang <[email protected]>

* update nan handling

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss func

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss

Signed-off-by: Zhilin Wang <[email protected]>

* fix type error raised in qa_dataset.py

Signed-off-by: Zhilin Wang <[email protected]>

* add error checking message

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update exp logging

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* limit number of negative samples

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused methods and style fix

Signed-off-by: Zhilin Wang <[email protected]>

* add more documentation

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* changes base on PR review

Signed-off-by: Zhilin Wang <[email protected]>

Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Fix bugs in indexed dataset exam script (#4325)

* fix the typo

Signed-off-by: Yi Dong <[email protected]>

* add neighbors option

Signed-off-by: Yi Dong <[email protected]>

* change the argument name

Signed-off-by: Yi Dong <[email protected]>

Co-authored-by: Micha Livne <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Adding docs for ASR SSL (#4303)

* Initial commit for SSL docs

Signed-off-by: Krishna Puvvada <[email protected]>

* ssl docs update-1

Signed-off-by: Krishna Puvvada <[email protected]>

* ssl docs update-2

Signed-off-by: Krishna Puvvada <[email protected]>

Co-authored-by: Krishna Puvvada <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Fuse grad division into async grad allreduce (#4327)

* O2 runs but O1 does not

Signed-off-by: ericharper <[email protected]>

* disable async for O1

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

* update async flag in configure_optimizers

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

* revert

Signed-off-by: ericharper <[email protected]>

* update _require if using async

Signed-off-by: ericharper <[email protected]>

* clean comments

Signed-off-by: ericharper <[email protected]>

* always all_reduce

Signed-off-by: ericharper <[email protected]>

* add async grad allreduce and chunk optimization to T5

* push reformatted files after style check

* set chunk size as 0 while async grad allreduce is off

* more experiments show that 125MB is a better default chunk size for most cases

* add grad_allreduce_chunk_size_mb for GPT-3

* at the end of each training step, wait until all async grad allreduce works are done

* replace individual allreduce work.wait() with a single dGPU evice synchroonization

* add code of fused_bias_geglu

* call fused_bias_geglu in ParallelMLP

* record the status of each allreduce work seems too much for perf

* add more comments

* push a reformatted file

* fix some bugs

* change biad_gelu_activation to bias_activation_fusion

* fix the setting of bias_actication_fusion for T5

* delete bias_gelu_fusion from T5 example config

* push reformatted files

* fuse grad scale with allreduce

* push reformatted files

* hto4h gemms fusion

* remove hto4h gemms fusion

* add grad_scale_ar_fusion into GPT-3

* push reformatted files

* push reformatted files

* rename grad_scale_ar_fusion to grad_div_ar_fusion

* disable bias_activation_fusion while activation is not geglu

* add bias_activation_fusion in yaml config file

* add bias_gelu_fusion in T5 config yaml file to pass CI test

* change bias_gelu_fusion to bias_activation_fusion for T5 CI test

* recover latest change

* add grad_div_ar_fusion in config yaml file

* remove a redundant float()

Co-authored-by: ericharper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Update container to 22.05 (#4329)

* update container to 22.05

Signed-off-by: ericharper <[email protected]>

* try adding safe directory

Signed-off-by: ericharper <[email protected]>

* try env var

Signed-off-by: ericharper <[email protected]>

* printenv

Signed-off-by: ericharper <[email protected]>

* try GIT_BRANCH

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

* remove dbug statements

Signed-off-by: ericharper <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Torchaudio installation fix (#4330)

* separate installer added

Signed-off-by: Aleksandr Laptev <[email protected]>

* apply suggestions, minor fixes

Signed-off-by: Aleksandr Laptev <[email protected]>

Co-authored-by: Aleksandr Laptev <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* [TTS] enforced pin_memory = True (#4341)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Merge r1.9.0 main (#4331)

* update branch

Signed-off-by: ericharper <[email protected]>

* update package info

Signed-off-by: ericharper <[email protected]>

* cleaned up TN/ ITN doc (#4119)

* cleaned up TN/ ITN doc

Signed-off-by: Yang Zhang <[email protected]>

* fix typo

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* Draft: Fix restoring from checkpoint for case when `model.common_dataset_parameters.label_vocab_dir` is provided (#4136)

* Fix restoring from checkpoint with label vocab dir

Signed-off-by: PeganovAnton <[email protected]>

* Add tests for various ways to pass label ids to model

Signed-off-by: PeganovAnton <[email protected]>

* Fix typo

Signed-off-by: PeganovAnton <[email protected]>

* Fix typo

Signed-off-by: PeganovAnton <[email protected]>

* Do not create tmp directory

Signed-off-by: PeganovAnton <[email protected]>

* Fix parameter name

Signed-off-by: PeganovAnton <[email protected]>

* finish cherry-pick op

Signed-off-by: PeganovAnton <[email protected]>

* Fix labels errors

Signed-off-by: PeganovAnton <[email protected]>

* Remove duplicate stage

Signed-off-by: PeganovAnton <[email protected]>

* Change target branch

Signed-off-by: PeganovAnton <[email protected]>

* fix doc (#4146)

Signed-off-by: Yang Zhang <[email protected]>

* Tacotron2 retrain (#4103)

* fix yaml

Signed-off-by: treacker <[email protected]>

* Fix for new TTSDataset class

Signed-off-by: treacker <[email protected]>

* added wandb logging

Signed-off-by: treacker <[email protected]>

* added wandb logging

Signed-off-by: treacker <[email protected]>

* fix numpy version

Signed-off-by: treacker <[email protected]>

* fix numpy version

Signed-off-by: treacker <[email protected]>

* inference fix

Signed-off-by: treacker <[email protected]>

* removed old code

Signed-off-by: treacker <[email protected]>

* updated parser logic

Signed-off-by: treacker <[email protected]>

* reverted version update

Signed-off-by: treacker <[email protected]>

* refactored parser logic

Signed-off-by: treacker <[email protected]>

* Updated Jenkinsfile

Signed-off-by: treacker <[email protected]>

* Refactored tutorial for Tacotron2

Signed-off-by: treacker <[email protected]>

* Made backward compatibility

Signed-off-by: treacker <[email protected]>

* Made backward compatibility

Signed-off-by: treacker <[email protected]>

* Update Jenkinsfile

Signed-off-by: treacker <[email protected]>

* Update tacotron.yaml

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* cleaned up TN/ ITN doc (#4119)

* cleaned up TN/ ITN doc

Signed-off-by: Yang Zhang <[email protected]>

* fix typo

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>
Signed-off-by: treacker <[email protected]>

* Check implicit grad acc in GLUE dataset building (#4123)

* Check implicit grad acc in GLUE dataset building

Signed-off-by: MaximumEntropy <[email protected]>

* Fix jenkins test for GLUE/XNLI

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Fixed jenkins

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>

* Multiprocess improvements (#4127)

* initial commit

Signed-off-by: nithinraok <[email protected]>

* start fix

Signed-off-by: nithinraok <[email protected]>

* improve multiprocessing speed while creating speaker dataset

Signed-off-by: nithinraok <[email protected]>

* updated scp to filelist

Signed-off-by: nithinraok <[email protected]>

* notebooks' link, typo and import  fix  (#4158)

* redo missing pr 4007

Signed-off-by: fayejf <[email protected]>

* remove extremely unreliable links

Signed-off-by: fayejf <[email protected]>

* update speaker docs (#4164)

* update speaker docs

Signed-off-by: nithinraok <[email protected]>

* chunks -> segments

Signed-off-by: nithinraok <[email protected]>

* Khz -> kHz

Signed-off-by: nithinraok <[email protected]>

* small fix (#4180)

Signed-off-by: fayejf <[email protected]>

* fix the server key value problem (#4196)

Signed-off-by: Yi Dong <[email protected]>

* Fix/punctuation/trainer required for setting test data (#4199)

* Draft of fix

Signed-off-by: PeganovAnton <[email protected]>

* Add warnings and replace globa_step with current_epoch

Signed-off-by: PeganovAnton <[email protected]>

* Small improvements to warnings

Signed-off-by: PeganovAnton <[email protected]>

* Error and warning messages improvements

Signed-off-by: PeganovAnton <[email protected]>

* Replace self.trainer with self._trainer

Signed-off-by: PeganovAnton <[email protected]>

* Update ContextNet version (#4207)

Signed-off-by: smajumdar <[email protected]>

* fix bugs for dialogue tutorial (#4211)

Signed-off-by: Zhilin Wang <[email protected]>

* Dialogue tutorial fix (#4214)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* Add docs for Thutmose Tagger (#4173)

* Add docs for Thutmose Tagger

Signed-off-by: Alexandra Antonova <[email protected]>

* add level in docs

Signed-off-by: Alexandra Antonova <[email protected]>

* delete folder to avoid error with running when folder exists from previous run

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>
Co-authored-by: ekmb <[email protected]>

* Dialogue tutorial fix (#4218)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously deleted files

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* Dialogue tutorial fix (#4221)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously deleted files

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* fix syntax error in ipynb-file (#4228)

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>

* fix json serialize (#4235)

Signed-off-by: Yi Dong <[email protected]>

* Prompt Learning Typo Fixes (#4238)

* Prompt tuning notebook typo fixes

Signed-off-by: Virginia Adams <[email protected]>

* Update tutorials.rst

* Update prompt_learning.rst

* Update prompt_learning.rst

* fixing bug 3642622 (#4250)

* fixing bug 3642622

Signed-off-by: Ghasem Pasandi <[email protected]>

* fixing bug 3642622

Signed-off-by: Ghasem Pasandi <[email protected]>

Co-authored-by: Ghasem Pasandi <[email protected]>

* fix broken link in the tutorial (#4257)

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>

* Typo fix, branch change, better download messagae (#4262)

Signed-off-by: Virginia Adams <[email protected]>

* Raise error if bicleaner is not installed in NMT Data preprocesing notebook (#4264)

* Raise error if bicleaner is not installed

Signed-off-by: MaximumEntropy <[email protected]>

* Clear cells

Signed-off-by: MaximumEntropy <[email protected]>

* Fix missing validation dataset, whitelist certain keywords for datasets (#4269)

* Fix missing validation dataset, whitelist certain keywords for datasets

Signed-off-by: smajumdar <[email protected]>

* Fix missing validation dataset, whitelist certain keywords for datasets

Signed-off-by: smajumdar <[email protected]>

* Update asr configs with num_workers and pin_memory (#4270)

Signed-off-by: smajumdar <[email protected]>

* Fix epoch end (#4265)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* Set Save on train end to false (#4274)

* Set Save on train end to false

Signed-off-by: Virginia Adams <[email protected]>

* Update prompt_learning.rst

* Update prompt_learning.rst

* Update YAML (#4261)

Signed-off-by: MaximumEntropy <[email protected]>

* Updated config to fix CI test OOM error (#4279)

* Updated config to fix CI test issue

Signed-off-by: Virginia Adams <[email protected]>

* Increased num workers

Signed-off-by: Virginia Adams <[email protected]>

* verbose k2 install, skip if failed (#4289)

Signed-off-by: Aleksandr Laptev <[email protected]>

Co-authored-by: Aleksandr Laptev <[email protected]>

* Changed total virtual prompt tokens (#4295)

* Changed total virtual prompt tokens

Signed-off-by: Virginia Adams <[email protected]>

* put number of workers back

Signed-off-by: Virginia Adams <[email protected]>

* upper bound lightning

Signed-off-by: ericharper <[email protected]>

* update branch

Signed-off-by: ericharper <[email protected]>

* update config

Signed-off-by: ericharper <[email protected]>

* remove duplicate test

Signed-off-by: ericharper <[email protected]>

* fix tn test cases

Signed-off-by: ericharper <[email protected]>

* add another safe.directory

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: PeganovAnton <[email protected]>
Co-authored-by: treacker <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Nithin Rao <[email protected]>
Co-authored-by: fayejf <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: bene-ges <[email protected]>
Co-authored-by: Alexandra Antonova <[email protected]>
Co-authored-by: ekmb <[email protected]>
Co-authored-by: Virginia Adams <[email protected]>
Co-authored-by: Ghasem <[email protected]>
Co-authored-by: Ghasem Pasandi <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* [TTS] Training Fastpitch on German text and phonemes and finetuning HiFi-GAN on predicted mels (#4266)

* initial commit

Signed-off-by: Akshit Arora <[email protected]>

* cleared notebook outputs

Signed-off-by: Akshit Arora <[email protected]>

* formatting errors

Signed-off-by: Akshit Arora <[email protected]>

* formatting

Signed-off-by: Akshit Arora <[email protected]>

* addressed comments

Signed-off-by: Akshit Arora <[email protected]>

* addressed comments on tutorial

Signed-off-by: Akshit Arora <[email protected]>

* updated tutorial

Signed-off-by: Akshit Arora <[email protected]>

* updated grammar and fastpitch description

Signed-off-by: Akshit Arora <[email protected]>

* updated with feedback

Signed-off-by: Akshit Arora <[email protected]>

* updated with feedback

Signed-off-by: Akshit Arora <[email protected]>

* removed unused imports

Signed-off-by: Akshit Arora <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Speedup the speech commands dataset processing script (#4347)

* Add multiprocessing support to the google speech commands dataset processing script

Signed-off-by: Shantanu Acharya <[email protected]>

* fix number of args error with __extract_all_files function

Signed-off-by: Shantanu Acharya <[email protected]>

* fix styling issues

Signed-off-by: Shantanu Acharya <[email protected]>

* fix bugs with silence set construction and update librosa output write to use soundfile write

Signed-off-by: Shantanu Acharya <[email protected]>

* add docstrings and return values in __construct_filepaths as dictionary

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* fix wrong requirement (#4349)

Signed-off-by: Yang Zhang <[email protected]>

Co-authored-by: Sandeep Subramanian <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Refactored path to manifest (#4251)

Signed-off-by: Evgeniy Shabalin <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* IPA support for TTS (#4310)

* IPA tokenizer and G2P untested draft

Signed-off-by: Jocelyn Huang <[email protected]>

* Add IPA CMUdict and new heteronyms list

Signed-off-by: Jocelyn Huang <[email protected]>

* Add draft FastPitch IPA config

Signed-off-by: Jocelyn Huang <[email protected]>

* Minor bugfixes for IPA training

Signed-off-by: Jocelyn Huang <[email protected]>

* Add phoneme_probability to IPA G2P

Signed-off-by: Jocelyn Huang <[email protected]>

* Updates to IPA FastPitch training config

Signed-off-by: Jocelyn Huang <[email protected]>

* Update IPA dict and heteronyms file

Signed-off-by: Jocelyn Huang <[email protected]>

* Adjust default lr for IPA FastPitch to 1e-3

Signed-off-by: Jocelyn Huang <[email protected]>

* Rename IPA CMUdict to reflect date

Signed-off-by: Jocelyn Huang <[email protected]>

* Add docstrings for IPA tokenizer and G2P, update CMUdict path for config

Signed-off-by: Jocelyn Huang <[email protected]>

* Fix IPA vocab ordering, add options to uppercase graphemes and remove stress symbols

Signed-off-by: Jocelyn Huang <[email protected]>

* Mark IPA classes as experimental

Signed-off-by: Jocelyn Huang <[email protected]>

* Update apostrophe-S cases

Signed-off-by: Jocelyn Huang <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Tn install (#4055)

* remove conda pynini requirement

Signed-off-by: Yang Zhang <[email protected]>

* remove remnants

Signed-off-by: Yang Zhang <[email protected]>

* merge with main

Signed-off-by: Yang Zhang <[email protected]>

* removing nlp collection dependency from text processing and thus breaking cyclyc imports

Signed-off-by: Yang Zhang <[email protected]>

* fix

Signed-off-by: Yang Zhang <[email protected]>

* fix wrong requirement

Signed-off-by: Yang Zhang <[email protected]>

* fix bug in vi

Signed-off-by: Yang Zhang <[email protected]>

* update jenkins folders

Signed-off-by: ekmb <[email protected]>

Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: ekmb <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* fix tutorial (#4352)

Signed-off-by: stevehuang52 <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* fix the post ln (#4350)

Signed-off-by: Yi Dong <[email protected]>

Co-authored-by: Eric Harper <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* [Fix] Hanging for Fully Randomized Bucketing (#4348)

* Update container to 22.05 (#4329)

* update container to 22.05

Signed-off-by: ericharper <[email protected]>

* try adding safe directory

Signed-off-by: ericharper <[email protected]>

* try env var

Signed-off-by: ericharper <[email protected]>

* printenv

Signed-off-by: ericharper <[email protected]>

* try GIT_BRANCH

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

* remove dbug statements

Signed-off-by: ericharper <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>

* Merge r1.9.0 main (#4331)

* update branch

Signed-off-by: ericharper <[email protected]>

* update package info

Signed-off-by: ericharper <[email protected]>

* cleaned up TN/ ITN doc (#4119)

* cleaned up TN/ ITN doc

Signed-off-by: Yang Zhang <[email protected]>

* fix typo

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* Draft: Fix restoring from checkpoint for case when `model.common_dataset_parameters.label_vocab_dir` is provided (#4136)

* Fix restoring from checkpoint with label vocab dir

Signed-off-by: PeganovAnton <[email protected]>

* Add tests for various ways to pass label ids to model

Signed-off-by: PeganovAnton <[email protected]>

* Fix typo

Signed-off-by: PeganovAnton <[email protected]>

* Fix typo

Signed-off-by: PeganovAnton <[email protected]>

* Do not create tmp directory

Signed-off-by: PeganovAnton <[email protected]>

* Fix parameter name

Signed-off-by: PeganovAnton <[email protected]>

* finish cherry-pick op

Signed-off-by: PeganovAnton <[email protected]>

* Fix labels errors

Signed-off-by: PeganovAnton <[email protected]>

* Remove duplicate stage

Signed-off-by: PeganovAnton <[email protected]>

* Change target branch

Signed-off-by: PeganovAnton <[email protected]>

* fix doc (#4146)

Signed-off-by: Yang Zhang <[email protected]>

* Tacotron2 retrain (#4103)

* fix yaml

Signed-off-by: treacker <[email protected]>

* Fix for new TTSDataset class

Signed-off-by: treacker <[email protected]>

* added wandb logging

Signed-off-by: treacker <[email protected]>

* added wandb logging

Signed-off-by: treacker <[email protected]>

* fix numpy version

Signed-off-by: treacker <[email protected]>

* fix numpy version

Signed-off-by: treacker <[email protected]>

* inference fix

Signed-off-by: treacker <[email protected]>

* removed old code

Signed-off-by: treacker <[email protected]>

* updated parser logic

Signed-off-by: treacker <[email protected]>

* reverted version update

Signed-off-by: treacker <[email protected]>

* refactored parser logic

Signed-off-by: treacker <[email protected]>

* Updated Jenkinsfile

Signed-off-by: treacker <[email protected]>

* Refactored tutorial for Tacotron2

Signed-off-by: treacker <[email protected]>

* Made backward compatibility

Signed-off-by: treacker <[email protected]>

* Made backward compatibility

Signed-off-by: treacker <[email protected]>

* Update Jenkinsfile

Signed-off-by: treacker <[email protected]>

* Update tacotron.yaml

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* cleaned up TN/ ITN doc (#4119)

* cleaned up TN/ ITN doc

Signed-off-by: Yang Zhang <[email protected]>

* fix typo

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>
Signed-off-by: treacker <[email protected]>

* Check implicit grad acc in GLUE dataset building (#4123)

* Check implicit grad acc in GLUE dataset building

Signed-off-by: MaximumEntropy <[email protected]>

* Fix jenkins test for GLUE/XNLI

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Fixed jenkins

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>

* Multiprocess improvements (#4127)

* initial commit

Signed-off-by: nithinraok <[email protected]>

* start fix

Signed-off-by: nithinraok <[email protected]>

* improve multiprocessing speed while creating speaker dataset

Signed-off-by: nithinraok <[email protected]>

* updated scp to filelist

Signed-off-by: nithinraok <[email protected]>

* notebooks' link, typo and import  fix  (#4158)

* redo missing pr 4007

Signed-off-by: fayejf <[email protected]>

* remove extremely unreliable links

Signed-off-by: fayejf <[email protected]>

* update speaker docs (#4164)

* update speaker docs

Signed-off-by: nithinraok <[email protected]>

* chunks -> segments

Signed-off-by: nithinraok <[email protected]>

* Khz -> kHz

Signed-off-by: nithinraok <[email protected]>

* small fix (#4180)

Signed-off-by: fayejf <[email protected]>

* fix the server key value problem (#4196)

Signed-off-by: Yi Dong <[email protected]>

* Fix/punctuation/trainer required for setting test data (#4199)

* Draft of fix

Signed-off-by: PeganovAnton <[email protected]>

* Add warnings and replace globa_step with current_epoch

Signed-off-by: PeganovAnton <[email protected]>

* Small improvements to warnings

Signed-off-by: PeganovAnton <[email protected]>

* Error and warning messages improvements

Signed-off-by: PeganovAnton <[email protected]>

* Replace self.trainer with self._trainer

Signed-off-by: PeganovAnton <[email protected]>

* Update ContextNet version (#4207)

Signed-off-by: smajumdar <[email protected]>

* fix bugs for dialogue tutorial (#4211)

Signed-off-by: Zhilin Wang <[email protected]>

* Dialogue tutorial fix (#4214)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* Add docs for Thutmose Tagger (#4173)

* Add docs for Thutmose Tagger

Signed-off-by: Alexandra Antonova <[email protected]>

* add level in docs

Signed-off-by: Alexandra Antonova <[email protected]>

* delete folder to avoid error with running when folder exists from previous run

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>
Co-authored-by: ekmb <[email protected]>

* Dialogue tutorial fix (#4218)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously deleted files

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* Dialogue tutorial fix (#4221)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously deleted files

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* fix syntax error in ipynb-file (#4228)

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>

* fix json serialize (#4235)

Signed-off-by: Yi Dong <[email protected]>

* Prompt Learning Typo Fixes (#4238)

* Prompt tuning notebook typo fixes

Signed-off-by: Virginia Adams <[email protected]>

* Update tutorials.rst

* Update prompt_learning.rst

* Update prompt_learning.rst

* fixing bug 3642622 (#4250)

* fixing bug 3642622

Signed-off-by: Ghasem Pasandi <[email protected]>

* fixing bug 3642622

Signed-off-by: Ghasem Pasandi <[email protected]>

Co-authored-by: Ghasem Pasandi <[email protected]>

* fix broken link in the tutorial (#4257)

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>

* Typo fix, branch change, better download messagae (#4262)

Signed-off-by: Virginia Adams <[email protected]>

* Raise error if bicleaner is not installed in NMT Data preprocesing notebook (#4264)

* Raise error if bicleaner is not installed

Signed-off-by: MaximumEntropy <[email protected]>

* Clear cells

Signed-off-by: MaximumEntropy <[email protected]>

* Fix missing validation dataset, whitelist certain keywords for datasets (#4269)

* Fix missing validation dataset, whitelist certain keywords for datasets

Signed-off-by: smajumdar <[email protected]>

* Fix missing validation dataset, whitelist certain keywords for datasets

Signed-off-by: smajumdar <[email protected]>

* Update asr configs with num_workers and pin_memory (#4270)

Signed-off-by: smajumdar <[email protected]>

* Fix epoch end (#4265)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* Set Save on train end to false (#4274)

* Set Save on train end to false

Signed-off-by: Virginia Adams <[email protected]>

* Update prompt_learning.rst

* Update prompt_learning.rst

* Update YAML (#4261)

Signed-off-by: MaximumEntropy <[email protected]>

* Updated config to fix CI test OOM error (#4279)

* Updated config to fix CI test issue

Signed-off-by: Virginia Adams <[email protected]>

* Increased num workers

Signed-off-by: Virginia Adams <[email protected]>

* verbose k2 install, skip if failed (#4289)

Signed-off-by: Aleksandr Laptev <[email protected]>

Co-authored-by: Aleksandr Laptev <[email protected]>

* Changed total virtual prompt tokens (#4295)

* Changed total virtual prompt tokens

Signed-off-by: Virginia Adams <[email protected]>

* put number of workers back

Signed-off-by: Virginia Adams <[email protected]>

* upper bound lightning

Signed-off-by: ericharper <[email protected]>

* update branch

Signed-off-by: ericharper <[email protected]>

* update config

Signed-off-by: ericharper <[email protected]>

* remove duplicate test

Signed-off-by: ericharper <[email protected]>

* fix tn test cases

Signed-off-by: ericharper <[email protected]>

* add another safe.directory

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: PeganovAnton <[email protected]>
Co-authored-by: treacker <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Nithin Rao <[email protected]>
Co-authored-by: fayejf <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: bene-ges <[email protected]>
Co-authored-by: Alexandra Antonova <[email protected]>
Co-authored-by: ekmb <[email protected]>
Co-authored-by: Virginia Adams <[email protected]>
Co-authored-by: Ghasem <[email protected]>
Co-authored-by: Ghasem Pasandi <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>

* fix full_randn bucket hang

Signed-off-by: stevehuang52 <[email protected]>

* remove unused variables

Signed-off-by: stevehuang52 <[email protected]>

Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: PeganovAnton <[email protected]>
Co-authored-by: treacker <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Nithin Rao <[email protected]>
Co-authored-by: fayejf <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: bene-ges <[email protected]>
Co-authored-by: Alexandra Antonova <[email protected]>
Co-authored-by: ekmb <[email protected]>
Co-authored-by: Virginia Adams <[email protected]>
Co-authored-by: Ghasem <[email protected]>
Co-authored-by: Ghasem Pasandi <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Bits of RADTTS support (#4343)

* Bits of RADTTS support

Signed-off-by: Boris Fomitchev <[email protected]>

* Fixed args mismatch

Signed-off-by: Boris Fomitchev <[email protected]>

* Style

Signed-off-by: Boris Fomitchev <[email protected]>

* Addressed review comments

Signed-off-by: Boris Fomitchev <[email protected]>

* More review comments

Signed-off-by: Boris Fomitchev <[email protected]>

Co-authored-by: Eric Harper <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Prompt Learning Pipeline Parallel (#4291)

* Added get_forward_output_and_loss_func and updated train/val steps

Signed-off-by: Virginia Adams <[email protected]>

* Added preprocess flag before prompt table/encoder access

Signed-off-by: Virginia Adams <[email protected]>

* Made two optimizer groups, one for frozen, one for soft prompt

Signed-off-by: Virginia Adams <[email protected]>

* Pipeline parallel working

Signed-off-by: Virginia Adams <[email protected]>

* Still figuring out setting lr/sched for one param group

Signed-off-by: Virginia Adams <[email protected]>

* Set betas to zero

Signed-off-by: Virginia Adams <[email protected]>

* Only unfreeze one sublayer with lr 0.0

Signed-off-by: Virginia Adams <[email protected]>

* Pipeline parallel working w/ one optimizer

Signed-off-by: Virginia Adams <[email protected]>

* Trying to fix Jenkins file

Signed-off-by: Virginia Adams <[email protected]>

* Trying to fix Jenkins file

Signed-off-by: Virginia Adams <[email protected]>

* Getting updated jenkins test to work

Signed-off-by: Virginia Adams <[email protected]>

* Getting updated jenkins test to work

Signed-off-by: Virginia Adams <[email protected]>

* added prompt learning tp and pp CI tests

Signed-off-by: Virginia Adams <[email protected]>

* Added amp_o2 model compatibility

Signed-off-by: Virginia Adams <[email protected]>

* Made CI test smaller

Signed-off-by: Virginia Adams <[email protected]>

* Still trying to get Jenkins to work

Signed-off-by: Virginia Adams <[email protected]>

* Still trying to get Jenkins to work

Signed-off-by: Virginia Adams <[email protected]>

* Temporarily moving prompt learning CI test to beginning

Signed-off-by: Virginia Adams <[email protected]>

* Changing the layer being unfrozen

Signed-off-by: Virginia Adams <[email protected]>

* debug jenkins

Signed-off-by: Virginia Adams <[email protected]>

* Move pp unfreeze to init

Signed-off-by: Virginia Adams <[email protected]>

* Try to make Jenkins test parallel

Signed-off-by: Virginia Adams <[email protected]>

* Fix python formatting

Signed-off-by: Virginia Adams <[email protected]>

* Moved prompt learning tests back to where they belong

Signed-off-by: Virginia Adams <[email protected]>

* add back checkpoint convertion CI test

Signed-off-by: Virginia Adams <[email protected]>

* Revert "add back checkpoint convertion CI test"

This reverts commit 61e2ffcdefe964c8e74b74d8c10906ae29f32b6d.

* Add back checkpoint conversion test

Signed-off-by: Virginia Adams <[email protected]>

* Setting requires grad to True everywhere

Signed-off-by: Virginia Adams <[email protected]>

* Updated config comments and simplified param group code

Signed-off-by: Virginia Adams <[email protected]>

* Added comment on frozen_model having lr=0.0

Signed-off-by: Virginia Adams <[email protected]>

* Added configure optimizers methods

Signed-off-by: Virginia Adams <[email protected]>

* Set amp_o2 to false

Signed-off-by: Virginia Adams <[email protected]>

* removed o2 code

Signed-off-by: Virginia Adams <[email protected]>

* Python formatting fix

Signed-off-by: Virginia Adams <[email protected]>

Co-authored-by: Eric Harper <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* auto switch conformer encoder adapter in_features (#4354)

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Dataloader, collector, loss and metric for multiscale diarization decoder  (#4187)

* First commit

Signed-off-by: Taejin Park <[email protected]>

* Checked funtionality and imports

Signed-off-by: Taejin Park <[email protected]>

* fixed import issues

Signed-off-by: Taejin Park <[email protected]>

* Removed the changed made by mistake

Signed-off-by: Taejin Park <[email protected]>

* Style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM errors 001

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM and style fix

Signed-off-by: Taejin Park <[email protected]>

* Changed docstrings

Signed-off-by: Taejin Park <[email protected]>

* LGTM again

Signed-off-by: Taejin Park <[email protected]>

* Removed unnecessary torch setting lines

Signed-off-by: Taejin Park <[email protected]>

* Style fix and isort

Signed-off-by: Taejin Park <[email protected]>

* jbalam-nv comments reflected

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected comments and created _diar_label.py

Signed-off-by: Taejin Park <[email protected]>

* Typo fix and style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed target_spks[0] index error

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* LGTM unused import IterDataset

Signed-off-by: Taejin Park <[email protected]>

* revert collection doc year

Signed-off-by: Taejin Park <[email protected]>

* Code format error in collections.py

Signed-off-by: Taejin Park <[email protected]>

* fix collections space format error

Signed-off-by: Taejin Park <[email protected]>

* merged main correctly

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected all comments and tested

Signed-off-by: Taejin Park <[email protected]>

* style fix and LGTM

Signed-off-by: Taejin Park <[email protected]>

* rttm_filepath to rttm_file and removed self included funcs, tested

Signed-off-by: Taejin Park <[email protected]>

Co-authored-by: Nithin Rao <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Add ASR CTC Decoding module (#4342)

* Initial commit

Signed-off-by: smajumdar <[email protected]>

* Full support for decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Temp

Signed-off-by: smajumdar <[email protected]>

* Fix labels of y_sequence

Signed-off-by: smajumdar <[email protected]>

* Set support for sentencepiece subword merging

Signed-off-by: smajumdar <[email protected]>

* Fix char and word based token merge alignment

Signed-off-by: smajumdar <[email protected]>

* Revert incorrect change

Signed-off-by: smajumdar <[email protected]>

* Update docstring

Signed-off-by: smajumdar <[email protected]>

* Improve compatibility with greedy tokens and log probs

Signed-off-by: smajumdar <[email protected]>

* Update scripts to use decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix decoding of ctc models

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Option to disable mp in VAD via num_workers=1 (#4317)

* Option to disable mp in VAD via num_workers=1

In certain environments python multiprocessing can deadlock. This adds a convenient version to disable by setting num_workers to 1.

Signed-off-by: Georg Kucsko <[email protected]>

* add none handling

Signed-off-by: Georg Kucsko <[email protected]>

* additional none handling

Signed-off-by: Georg Kucsko <[email protected]>

Co-authored-by: fayejf <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* remove redundant bias expand (#4382)

* remove redundant bias expand

Signed-off-by: Xiaowei Ren <[email protected]>

* delete redundant code

Signed-off-by: Xiaowei Ren <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Add option for specifying wandb save_dir from config (#4379)

* give option to user to specify wandb save dir via config

Signed-off-by: Shantanu Acharya <[email protected]>

* create save_dir directory for wandb logger if not exists

Signed-off-by: Shantanu Acharya <[email protected]>

* update save_dir get method with a default value

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* [Bugfix][TTS] wrong order of returned tuple for general_collate_fn. (#4388)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Merge r1.10.0 main (#4398)

* update branch

Signed-off-by: ericharper <[email protected]>

* Set headscale false (#4364)

Signed-off-by: MaximumEntropy <[email protected]>

* Add wandb as dependency (#4365)

Signed-off-by: smajumdar <[email protected]>

* Raise trainer error (#4356)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Set headscale false (#4364) (#4366)

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: smajumdar <[email protected]>

* Finetuning changes for BART (#4003)

* Temp

Signed-off-by: MaximumEntropy <[email protected]>

* Checkpoint converter to nemo for bart

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Make position embedding expansion specific to a batch to avoid checkpoint size mismatches (#4357)

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Fix logging warning

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Fix electronic bug, new time ITN rule (#4355)

* fix electronic bug

Signed-off-by: ekmb <[email protected]>

* add new itn time rule

Signed-off-by: ekmb <[email protected]>

* revert domain changes

Signed-off-by: ekmb <[email protected]>

* remove repetition

Signed-off-by: ekmb <[email protected]>

* Correct support for dataclasses in default module dim (#4372)

* Correct support for dataclasses in default module dim

Signed-off-by: smajumdar <[email protected]>

* Fix path for save of results

Signed-off-by: smajumdar <[email protected]>

* fix pad id bug (#4377)

Signed-off-by: Yi Dong <[email protected]>

* Question answering bug fix (#4381)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assistant data processsor

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins for SGDGEN local checkpoint

Signed-off-by: Zhilin Wang <[email protected]>

* update style

Signed-off-by: Zhilin Wang <[email protected]>

* use local vocab file for Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* patch for Jenkins CI using local file

Signed-off-by: Zhilin Wang <[email protected]>

* add slot filling prediction and metrics

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused code

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

…
hainan-xv pushed a commit to hainan-xv/NeMo that referenced this pull request Nov 29, 2022
* give option to user to specify wandb save dir via config

Signed-off-by: Shantanu Acharya <[email protected]>

* create save_dir directory for wandb logger if not exists

Signed-off-by: Shantanu Acharya <[email protected]>

* update save_dir get method with a default value

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>
hainan-xv pushed a commit to hainan-xv/NeMo that referenced this pull request Nov 29, 2022
* Dataloader, collector, loss and metric for multiscale diarization decoder  (NVIDIA#4187)

* First commit

Signed-off-by: Taejin Park <[email protected]>

* Checked funtionality and imports

Signed-off-by: Taejin Park <[email protected]>

* fixed import issues

Signed-off-by: Taejin Park <[email protected]>

* Removed the changed made by mistake

Signed-off-by: Taejin Park <[email protected]>

* Style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM errors 001

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM and style fix

Signed-off-by: Taejin Park <[email protected]>

* Changed docstrings

Signed-off-by: Taejin Park <[email protected]>

* LGTM again

Signed-off-by: Taejin Park <[email protected]>

* Removed unnecessary torch setting lines

Signed-off-by: Taejin Park <[email protected]>

* Style fix and isort

Signed-off-by: Taejin Park <[email protected]>

* jbalam-nv comments reflected

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected comments and created _diar_label.py

Signed-off-by: Taejin Park <[email protected]>

* Typo fix and style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed target_spks[0] index error

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* LGTM unused import IterDataset

Signed-off-by: Taejin Park <[email protected]>

* revert collection doc year

Signed-off-by: Taejin Park <[email protected]>

* Code format error in collections.py

Signed-off-by: Taejin Park <[email protected]>

* fix collections space format error

Signed-off-by: Taejin Park <[email protected]>

* merged main correctly

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected all comments and tested

Signed-off-by: Taejin Park <[email protected]>

* style fix and LGTM

Signed-off-by: Taejin Park <[email protected]>

* rttm_filepath to rttm_file and removed self included funcs, tested

Signed-off-by: Taejin Park <[email protected]>

Co-authored-by: Nithin Rao <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* removed references to data_dir

Signed-off-by: Matvei Novikov <[email protected]>

* added missing parameters to data preparation script

Signed-off-by: Matvei Novikov <[email protected]>

* removed unnecessary file extension check

Signed-off-by: Matvei Novikov <[email protected]>

* Add ASR CTC Decoding module (NVIDIA#4342)

* Initial commit

Signed-off-by: smajumdar <[email protected]>

* Full support for decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Temp

Signed-off-by: smajumdar <[email protected]>

* Fix labels of y_sequence

Signed-off-by: smajumdar <[email protected]>

* Set support for sentencepiece subword merging

Signed-off-by: smajumdar <[email protected]>

* Fix char and word based token merge alignment

Signed-off-by: smajumdar <[email protected]>

* Revert incorrect change

Signed-off-by: smajumdar <[email protected]>

* Update docstring

Signed-off-by: smajumdar <[email protected]>

* Improve compatibility with greedy tokens and log probs

Signed-off-by: smajumdar <[email protected]>

* Update scripts to use decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix decoding of ctc models

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Option to disable mp in VAD via num_workers=1 (NVIDIA#4317)

* Option to disable mp in VAD via num_workers=1

In certain environments python multiprocessing can deadlock. This adds a convenient version to disable by setting num_workers to 1.

Signed-off-by: Georg Kucsko <[email protected]>

* add none handling

Signed-off-by: Georg Kucsko <[email protected]>

* additional none handling

Signed-off-by: Georg Kucsko <[email protected]>

Co-authored-by: fayejf <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* remove redundant bias expand (NVIDIA#4382)

* remove redundant bias expand

Signed-off-by: Xiaowei Ren <[email protected]>

* delete redundant code

Signed-off-by: Xiaowei Ren <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* fixed style

Signed-off-by: Matvei Novikov <[email protected]>

* Add option for specifying wandb save_dir from config (NVIDIA#4379)

* give option to user to specify wandb save dir via config

Signed-off-by: Shantanu Acharya <[email protected]>

* create save_dir directory for wandb logger if not exists

Signed-off-by: Shantanu Acharya <[email protected]>

* update save_dir get method with a default value

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (NVIDIA#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* [Bugfix][TTS] wrong order of returned tuple for general_collate_fn. (NVIDIA#4388)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Merge r1.10.0 main (NVIDIA#4398)

* update branch

Signed-off-by: ericharper <[email protected]>

* Set headscale false (NVIDIA#4364)

Signed-off-by: MaximumEntropy <[email protected]>

* Add wandb as dependency (NVIDIA#4365)

Signed-off-by: smajumdar <[email protected]>

* Raise trainer error (NVIDIA#4356)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Set headscale false (NVIDIA#4364) (NVIDIA#4366)

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: smajumdar <[email protected]>

* Finetuning changes for BART (NVIDIA#4003)

* Temp

Signed-off-by: MaximumEntropy <[email protected]>

* Checkpoint converter to nemo for bart

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Make position embedding expansion specific to a batch to avoid checkpoint size mismatches (NVIDIA#4357)

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Fix logging warning

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Fix electronic bug, new time ITN rule (NVIDIA#4355)

* fix electronic bug

Signed-off-by: ekmb <[email protected]>

* add new itn time rule

Signed-off-by: ekmb <[email protected]>

* revert domain changes

Signed-off-by: ekmb <[email protected]>

* remove repetition

Signed-off-by: ekmb <[email protected]>

* Correct support for dataclasses in default module dim (NVIDIA#4372)

* Correct support for dataclasses in default module dim

Signed-off-by: smajumdar <[email protected]>

* Fix path for save of results

Signed-off-by: smajumdar <[email protected]>

* fix pad id bug (NVIDIA#4377)

Signed-off-by: Yi Dong <[email protected]>

* Question answering bug fix (NVIDIA#4381)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assistant data processsor

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins for SGDGEN local checkpoint

Signed-off-by: Zhilin Wang <[email protected]>

* update style

Signed-off-by: Zhilin Wang <[email protected]>

* use local vocab file for Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* patch for Jenkins CI using local file

Signed-off-by: Zhilin Wang <[email protected]>

* add slot filling prediction and metrics

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused code

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* refactor metrics code out of Dialogue GPT Model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate backward compatible support for IntentSlotClassificationModel (bert model)

Signed-off-by: Zhilin Wang <[email protected]>

* save prediction file for IntentSlotClassification

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue gpt model training for megatron gpt

Signed-off-by: Zhilin Wang <[email protected]>

* remove batch generate for HF GPT2, which causes lower performance

Signed-off-by: Zhilin Wang <[email protected]>

* add few shot capability to dialogue gpt model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile and remove unused import

Signed-off-by: Zhilin Wang <[email protected]>

* update code description and clarity

Signed-off-by: Zhilin Wang <[email protected]>

* address PR comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate compatibility with ZeroShotIntentModel

Signed-off-by: Zhilin Wang <[email protected]>

* rename folder to dialogue due to increased scope and further refactor for clarity

Signed-off-by: Zhilin Wang <[email protected]>

* added dialogue GPT for sequence generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* add CI test for DialogueGPTGenerationModel

Signed-off-by: Zhilin Wang <[email protected]>

* integrate DialogueS2SGenerationModel for generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* modify huggingface utils to support HF t5/BART models

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* fix bleu metric style

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* update based on PR NVIDIA#3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 2 based on PR NVIDIA#3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 3 based on PR NVIDIA#3893

Signed-off-by: Zhilin Wang <[email protected]>

* integrate sgd generation based on user user utterance and system slot-values to generate system utterance

Signed-off-by: Zhilin Wang <[email protected]>

* add validation model saving capabilities

Signed-off-by: Zhilin Wang <[email protected]>

* cleaned up code for SGD Based Answer extender

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue Generation CI

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* fix Jenkins CI issue"

Signed-off-by: Zhilin Wang <[email protected]>

* add support for design dataset

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary imports

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support megatron for dialogue_s2s_generation_model

Signed-off-by: Zhilin Wang <[email protected]>

* reduce loaded samples in MSMarcoDataProcessor to 64 when cfg.model.dataset.debug_mode=True

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update CI

Signed-off-by: Zhilin Wang <[email protected]>

* update checkpoint and predictions filename to include epoch number

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate HF BART MNLI into zero shot intent model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Nearest Neighbour Model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* refactor Dialogue SGD Data Processor to make interface for models cleaner

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue S2S Generation model for DialogueSGDDataProcessor interface

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support sgd and drive thru datasets by zero shot model and nearest neighbour model

Signed-off-by: Zhilin Wang <[email protected]>

* add prediction saving code to nearest neighbour and zero shot intent models

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo in sgd data processor

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Mellon QA Data Processor

Signed-off-by: Zhilin Wang <[email protected]>

* update mellon qa

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue.py to remove outdated info

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* add dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* address review comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix for cfg

Signed-off-by: Zhilin Wang <[email protected]>

* make dependency on apex optional

Signed-off-by: Zhilin Wang <[email protected]>

* change NLPDDPluggin calling logic to make it possible to run without apex

Signed-off-by: Zhilin Wang <[email protected]>

* add first draft of tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* reduce ms marco size by removing lines without wellFormedAnswers

Signed-off-by: Zhilin Wang <[email protected]>

* address pr comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update colab tutorial link in dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* include unit test and some refactor to facilitate unit test

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* address pr issues

Signed-off-by: Zhilin Wang <[email protected]>

* remove typos in dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* support larger files for question answering

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary artifacts to reduce memory use

Signed-off-by: Zhilin Wang <[email protected]>

* put 0 tensor to device

Signed-off-by: Zhilin Wang <[email protected]>

* update link within dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously delete files

Signed-off-by: Zhilin Wang <[email protected]>

* update error handling when loss = nan

Signed-off-by: Zhilin Wang <[email protected]>

* update nan handling

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss func

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss

Signed-off-by: Zhilin Wang <[email protected]>

* fix type error raised in qa_dataset.py

Signed-off-by: Zhilin Wang <[email protected]>

* add error checking message

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update exp logging

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* limit number of negative samples

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused methods and style fix

Signed-off-by: Zhilin Wang <[email protected]>

* add more documentation

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* changes base on PR review

Signed-off-by: Zhilin Wang <[email protected]>

* set wandb logger falseby default

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

* style fix

* correct typo

* style fix

* style fix

Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>

* Fix ASR Typos in tutorials (NVIDIA#4384)

* Fix typos

Signed-off-by: smajumdar <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (NVIDIA#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
(cherry picked from commit 0322b15)

Co-authored-by: Travis Bartley <[email protected]>

* Add Docs for NeMo Adapters (NVIDIA#4369)

Signed-off-by: smajumdar <[email protected]>

* Update NeMo docs (NVIDIA#4397)

Signed-off-by: smajumdar <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* update branch

Signed-off-by: ericharper <[email protected]>

* remove Copy of

Signed-off-by: ericharper <[email protected]>

Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Micha Livne <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Travis Bartley <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* [bugfix][TTS] pitch, voiced_mask, prob_voiced have the same values. (NVIDIA#4392)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Fixing import error in some cases (NVIDIA#4401)

Signed-off-by: Boris Fomitchev <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Fixing bugs in calling method ctc_decoder_predictions_tensor. (NVIDIA#4414)

* updated ctc decoding calls.

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>

* fixed the ones for timestamp_utils.py

Signed-off-by: Vahid <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

* Update with new conformer checkpoints. (NVIDIA#4417)

Signed-off-by: Matvei Novikov <[email protected]>

* [TTS] add static method decorator. (NVIDIA#4443)

* [TTS] add static method decorator.

Signed-off-by: Xuesong Yang <[email protected]>

* remove protect prefix

Signed-off-by: Xuesong Yang <[email protected]>

* fixed style error

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Matvei Novikov <[email protected]>

Co-authored-by: Taejin Park <[email protected]>
Co-authored-by: Nithin Rao <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Georg Kucsko <[email protected]>
Co-authored-by: fayejf <[email protected]>
Co-authored-by: Xiaowei Ren <[email protected]>
Co-authored-by: Shantanu Acharya <[email protected]>
Co-authored-by: Travis Bartley <[email protected]>
Co-authored-by: tbartley94 <[email protected]>
Co-authored-by: Xuesong Yang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Micha Livne <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Boris Fomitchev <[email protected]>
Co-authored-by: Vahid Noroozi <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>
hainan-xv added a commit to hainan-xv/NeMo that referenced this pull request Nov 29, 2022
* stateless RNNT working

Signed-off-by: Hainan Xu <[email protected]>

* batch decode working

Signed-off-by: Hainan Xu <[email protected]>

* working backup

Signed-off-by: Hainan Xu <[email protected]>

* good working version

Signed-off-by: Hainan Xu <[email protected]>

* temporarily make norm layer have affine

Signed-off-by: Hainan Xu <[email protected]>

* temp

Signed-off-by: Hainan Xu <[email protected]>

* temp

Signed-off-by: Hainan Xu <[email protected]>

* [TTS] add staticmethod decoration for BetaBinomialInterpolator (#4319)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* [TTS] remove redundant lines and declare global variables and capture (#4320)

exception of non-supported windows.

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Finetune T5 on the prefix-lm objective (#4328)

* Add script and yaml config

Signed-off-by: MaximumEntropy <[email protected]>

* Fix yaml config

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Update yaml to remove hardcoded model path

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Oleksii Kuchaiev <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Fuse bias with geglu in ParallelMLP (#4213)

* add code of fused_bias_geglu

* call fused_bias_geglu in ParallelMLP

* fix some bugs

* change biad_gelu_activation to bias_activation_fusion

* fix the setting of bias_actication_fusion for T5

* delete bias_gelu_fusion from T5 example config

* push reformatted files

* hto4h gemms fusion

* remove hto4h gemms fusion

* push reformatted files

* disable bias_activation_fusion while activation is not geglu

* add bias_activation_fusion in yaml config file

* add bias_gelu_fusion in T5 config yaml file to pass CI test

* change bias_gelu_fusion to bias_activation_fusion for T5 CI test

* recover latest change

Co-authored-by: Sandeep Subramanian <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Support larger datasets for question answering  (#4205)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assistant data processsor

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins for SGDGEN local checkpoint

Signed-off-by: Zhilin Wang <[email protected]>

* update style

Signed-off-by: Zhilin Wang <[email protected]>

* use local vocab file for Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* patch for Jenkins CI using local file

Signed-off-by: Zhilin Wang <[email protected]>

* add slot filling prediction and metrics

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused code

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* refactor metrics code out of Dialogue GPT Model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate backward compatible support for IntentSlotClassificationModel (bert model)

Signed-off-by: Zhilin Wang <[email protected]>

* save prediction file for IntentSlotClassification

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue gpt model training for megatron gpt

Signed-off-by: Zhilin Wang <[email protected]>

* remove batch generate for HF GPT2, which causes lower performance

Signed-off-by: Zhilin Wang <[email protected]>

* add few shot capability to dialogue gpt model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile and remove unused import

Signed-off-by: Zhilin Wang <[email protected]>

* update code description and clarity

Signed-off-by: Zhilin Wang <[email protected]>

* address PR comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate compatibility with ZeroShotIntentModel

Signed-off-by: Zhilin Wang <[email protected]>

* rename folder to dialogue due to increased scope and further refactor for clarity

Signed-off-by: Zhilin Wang <[email protected]>

* added dialogue GPT for sequence generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* add CI test for DialogueGPTGenerationModel

Signed-off-by: Zhilin Wang <[email protected]>

* integrate DialogueS2SGenerationModel for generation task (e.g. answer extender)

Signed-off-by: Zhilin Wang <[email protected]>

* modify huggingface utils to support HF t5/BART models

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* fix bleu metric style

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* debug bleu metric

Signed-off-by: Zhilin Wang <[email protected]>

* update based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 2 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* update 3 based on PR #3893

Signed-off-by: Zhilin Wang <[email protected]>

* integrate sgd generation based on user user utterance and system slot-values to generate system utterance

Signed-off-by: Zhilin Wang <[email protected]>

* add validation model saving capabilities

Signed-off-by: Zhilin Wang <[email protected]>

* cleaned up code for SGD Based Answer extender

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue Generation CI

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* fix Jenkins CI issue"

Signed-off-by: Zhilin Wang <[email protected]>

* add support for design dataset

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary imports

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support megatron for dialogue_s2s_generation_model

Signed-off-by: Zhilin Wang <[email protected]>

* reduce loaded samples in MSMarcoDataProcessor to 64 when cfg.model.dataset.debug_mode=True

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update CI

Signed-off-by: Zhilin Wang <[email protected]>

* update checkpoint and predictions filename to include epoch number

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* integrate HF BART MNLI into zero shot intent model

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Nearest Neighbour Model

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* refactor Dialogue SGD Data Processor to make interface for models cleaner

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update Dialogue S2S Generation model for DialogueSGDDataProcessor interface

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* update jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* support sgd and drive thru datasets by zero shot model and nearest neighbour model

Signed-off-by: Zhilin Wang <[email protected]>

* add prediction saving code to nearest neighbour and zero shot intent models

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo in sgd data processor

Signed-off-by: Zhilin Wang <[email protected]>

* integrate Dialogue Mellon QA Data Processor

Signed-off-by: Zhilin Wang <[email protected]>

* update mellon qa

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue.py to remove outdated info

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* update dialogue_config.yaml

Signed-off-by: Zhilin Wang <[email protected]>

* add dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* address review comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix for cfg

Signed-off-by: Zhilin Wang <[email protected]>

* make dependency on apex optional

Signed-off-by: Zhilin Wang <[email protected]>

* change NLPDDPluggin calling logic to make it possible to run without apex

Signed-off-by: Zhilin Wang <[email protected]>

* add first draft of tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* reduce ms marco size by removing lines without wellFormedAnswers

Signed-off-by: Zhilin Wang <[email protected]>

* address pr comments

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update colab tutorial link in dialogue docs

Signed-off-by: Zhilin Wang <[email protected]>

* include unit test and some refactor to facilitate unit test

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* address pr issues

Signed-off-by: Zhilin Wang <[email protected]>

* remove typos in dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* support larger files for question answering

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* remove unnecessary artifacts to reduce memory use

Signed-off-by: Zhilin Wang <[email protected]>

* put 0 tensor to device

Signed-off-by: Zhilin Wang <[email protected]>

* update link within dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously delete files

Signed-off-by: Zhilin Wang <[email protected]>

* update error handling when loss = nan

Signed-off-by: Zhilin Wang <[email protected]>

* update nan handling

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss func

Signed-off-by: Zhilin Wang <[email protected]>

* update spanning loss

Signed-off-by: Zhilin Wang <[email protected]>

* fix type error raised in qa_dataset.py

Signed-off-by: Zhilin Wang <[email protected]>

* add error checking message

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* revert back to float32

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update exp logging

Signed-off-by: Zhilin Wang <[email protected]>

* update error msgs

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* update loading of large file from pickle to json

Signed-off-by: Zhilin Wang <[email protected]>

* limit number of negative samples

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* revert post processing

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused methods and style fix

Signed-off-by: Zhilin Wang <[email protected]>

* add more documentation

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused imports

Signed-off-by: Zhilin Wang <[email protected]>

* changes base on PR review

Signed-off-by: Zhilin Wang <[email protected]>

Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Oleksii Kuchaiev <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Fix bugs in indexed dataset exam script (#4325)

* fix the typo

Signed-off-by: Yi Dong <[email protected]>

* add neighbors option

Signed-off-by: Yi Dong <[email protected]>

* change the argument name

Signed-off-by: Yi Dong <[email protected]>

Co-authored-by: Micha Livne <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Adding docs for ASR SSL (#4303)

* Initial commit for SSL docs

Signed-off-by: Krishna Puvvada <[email protected]>

* ssl docs update-1

Signed-off-by: Krishna Puvvada <[email protected]>

* ssl docs update-2

Signed-off-by: Krishna Puvvada <[email protected]>

Co-authored-by: Krishna Puvvada <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Fuse grad division into async grad allreduce (#4327)

* O2 runs but O1 does not

Signed-off-by: ericharper <[email protected]>

* disable async for O1

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

* update async flag in configure_optimizers

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

* revert

Signed-off-by: ericharper <[email protected]>

* update _require if using async

Signed-off-by: ericharper <[email protected]>

* clean comments

Signed-off-by: ericharper <[email protected]>

* always all_reduce

Signed-off-by: ericharper <[email protected]>

* add async grad allreduce and chunk optimization to T5

* push reformatted files after style check

* set chunk size as 0 while async grad allreduce is off

* more experiments show that 125MB is a better default chunk size for most cases

* add grad_allreduce_chunk_size_mb for GPT-3

* at the end of each training step, wait until all async grad allreduce works are done

* replace individual allreduce work.wait() with a single dGPU evice synchroonization

* add code of fused_bias_geglu

* call fused_bias_geglu in ParallelMLP

* record the status of each allreduce work seems too much for perf

* add more comments

* push a reformatted file

* fix some bugs

* change biad_gelu_activation to bias_activation_fusion

* fix the setting of bias_actication_fusion for T5

* delete bias_gelu_fusion from T5 example config

* push reformatted files

* fuse grad scale with allreduce

* push reformatted files

* hto4h gemms fusion

* remove hto4h gemms fusion

* add grad_scale_ar_fusion into GPT-3

* push reformatted files

* push reformatted files

* rename grad_scale_ar_fusion to grad_div_ar_fusion

* disable bias_activation_fusion while activation is not geglu

* add bias_activation_fusion in yaml config file

* add bias_gelu_fusion in T5 config yaml file to pass CI test

* change bias_gelu_fusion to bias_activation_fusion for T5 CI test

* recover latest change

* add grad_div_ar_fusion in config yaml file

* remove a redundant float()

Co-authored-by: ericharper <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Update container to 22.05 (#4329)

* update container to 22.05

Signed-off-by: ericharper <[email protected]>

* try adding safe directory

Signed-off-by: ericharper <[email protected]>

* try env var

Signed-off-by: ericharper <[email protected]>

* printenv

Signed-off-by: ericharper <[email protected]>

* try GIT_BRANCH

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

* remove dbug statements

Signed-off-by: ericharper <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Torchaudio installation fix (#4330)

* separate installer added

Signed-off-by: Aleksandr Laptev <[email protected]>

* apply suggestions, minor fixes

Signed-off-by: Aleksandr Laptev <[email protected]>

Co-authored-by: Aleksandr Laptev <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* [TTS] enforced pin_memory = True (#4341)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Merge r1.9.0 main (#4331)

* update branch

Signed-off-by: ericharper <[email protected]>

* update package info

Signed-off-by: ericharper <[email protected]>

* cleaned up TN/ ITN doc (#4119)

* cleaned up TN/ ITN doc

Signed-off-by: Yang Zhang <[email protected]>

* fix typo

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* Draft: Fix restoring from checkpoint for case when `model.common_dataset_parameters.label_vocab_dir` is provided (#4136)

* Fix restoring from checkpoint with label vocab dir

Signed-off-by: PeganovAnton <[email protected]>

* Add tests for various ways to pass label ids to model

Signed-off-by: PeganovAnton <[email protected]>

* Fix typo

Signed-off-by: PeganovAnton <[email protected]>

* Fix typo

Signed-off-by: PeganovAnton <[email protected]>

* Do not create tmp directory

Signed-off-by: PeganovAnton <[email protected]>

* Fix parameter name

Signed-off-by: PeganovAnton <[email protected]>

* finish cherry-pick op

Signed-off-by: PeganovAnton <[email protected]>

* Fix labels errors

Signed-off-by: PeganovAnton <[email protected]>

* Remove duplicate stage

Signed-off-by: PeganovAnton <[email protected]>

* Change target branch

Signed-off-by: PeganovAnton <[email protected]>

* fix doc (#4146)

Signed-off-by: Yang Zhang <[email protected]>

* Tacotron2 retrain (#4103)

* fix yaml

Signed-off-by: treacker <[email protected]>

* Fix for new TTSDataset class

Signed-off-by: treacker <[email protected]>

* added wandb logging

Signed-off-by: treacker <[email protected]>

* added wandb logging

Signed-off-by: treacker <[email protected]>

* fix numpy version

Signed-off-by: treacker <[email protected]>

* fix numpy version

Signed-off-by: treacker <[email protected]>

* inference fix

Signed-off-by: treacker <[email protected]>

* removed old code

Signed-off-by: treacker <[email protected]>

* updated parser logic

Signed-off-by: treacker <[email protected]>

* reverted version update

Signed-off-by: treacker <[email protected]>

* refactored parser logic

Signed-off-by: treacker <[email protected]>

* Updated Jenkinsfile

Signed-off-by: treacker <[email protected]>

* Refactored tutorial for Tacotron2

Signed-off-by: treacker <[email protected]>

* Made backward compatibility

Signed-off-by: treacker <[email protected]>

* Made backward compatibility

Signed-off-by: treacker <[email protected]>

* Update Jenkinsfile

Signed-off-by: treacker <[email protected]>

* Update tacotron.yaml

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* cleaned up TN/ ITN doc (#4119)

* cleaned up TN/ ITN doc

Signed-off-by: Yang Zhang <[email protected]>

* fix typo

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>
Signed-off-by: treacker <[email protected]>

* Check implicit grad acc in GLUE dataset building (#4123)

* Check implicit grad acc in GLUE dataset building

Signed-off-by: MaximumEntropy <[email protected]>

* Fix jenkins test for GLUE/XNLI

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Fixed jenkins

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>

* Multiprocess improvements (#4127)

* initial commit

Signed-off-by: nithinraok <[email protected]>

* start fix

Signed-off-by: nithinraok <[email protected]>

* improve multiprocessing speed while creating speaker dataset

Signed-off-by: nithinraok <[email protected]>

* updated scp to filelist

Signed-off-by: nithinraok <[email protected]>

* notebooks' link, typo and import  fix  (#4158)

* redo missing pr 4007

Signed-off-by: fayejf <[email protected]>

* remove extremely unreliable links

Signed-off-by: fayejf <[email protected]>

* update speaker docs (#4164)

* update speaker docs

Signed-off-by: nithinraok <[email protected]>

* chunks -> segments

Signed-off-by: nithinraok <[email protected]>

* Khz -> kHz

Signed-off-by: nithinraok <[email protected]>

* small fix (#4180)

Signed-off-by: fayejf <[email protected]>

* fix the server key value problem (#4196)

Signed-off-by: Yi Dong <[email protected]>

* Fix/punctuation/trainer required for setting test data (#4199)

* Draft of fix

Signed-off-by: PeganovAnton <[email protected]>

* Add warnings and replace globa_step with current_epoch

Signed-off-by: PeganovAnton <[email protected]>

* Small improvements to warnings

Signed-off-by: PeganovAnton <[email protected]>

* Error and warning messages improvements

Signed-off-by: PeganovAnton <[email protected]>

* Replace self.trainer with self._trainer

Signed-off-by: PeganovAnton <[email protected]>

* Update ContextNet version (#4207)

Signed-off-by: smajumdar <[email protected]>

* fix bugs for dialogue tutorial (#4211)

Signed-off-by: Zhilin Wang <[email protected]>

* Dialogue tutorial fix (#4214)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* Add docs for Thutmose Tagger (#4173)

* Add docs for Thutmose Tagger

Signed-off-by: Alexandra Antonova <[email protected]>

* add level in docs

Signed-off-by: Alexandra Antonova <[email protected]>

* delete folder to avoid error with running when folder exists from previous run

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>
Co-authored-by: ekmb <[email protected]>

* Dialogue tutorial fix (#4218)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously deleted files

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* Dialogue tutorial fix (#4221)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously deleted files

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* fix syntax error in ipynb-file (#4228)

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>

* fix json serialize (#4235)

Signed-off-by: Yi Dong <[email protected]>

* Prompt Learning Typo Fixes (#4238)

* Prompt tuning notebook typo fixes

Signed-off-by: Virginia Adams <[email protected]>

* Update tutorials.rst

* Update prompt_learning.rst

* Update prompt_learning.rst

* fixing bug 3642622 (#4250)

* fixing bug 3642622

Signed-off-by: Ghasem Pasandi <[email protected]>

* fixing bug 3642622

Signed-off-by: Ghasem Pasandi <[email protected]>

Co-authored-by: Ghasem Pasandi <[email protected]>

* fix broken link in the tutorial (#4257)

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>

* Typo fix, branch change, better download messagae (#4262)

Signed-off-by: Virginia Adams <[email protected]>

* Raise error if bicleaner is not installed in NMT Data preprocesing notebook (#4264)

* Raise error if bicleaner is not installed

Signed-off-by: MaximumEntropy <[email protected]>

* Clear cells

Signed-off-by: MaximumEntropy <[email protected]>

* Fix missing validation dataset, whitelist certain keywords for datasets (#4269)

* Fix missing validation dataset, whitelist certain keywords for datasets

Signed-off-by: smajumdar <[email protected]>

* Fix missing validation dataset, whitelist certain keywords for datasets

Signed-off-by: smajumdar <[email protected]>

* Update asr configs with num_workers and pin_memory (#4270)

Signed-off-by: smajumdar <[email protected]>

* Fix epoch end (#4265)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* Set Save on train end to false (#4274)

* Set Save on train end to false

Signed-off-by: Virginia Adams <[email protected]>

* Update prompt_learning.rst

* Update prompt_learning.rst

* Update YAML (#4261)

Signed-off-by: MaximumEntropy <[email protected]>

* Updated config to fix CI test OOM error (#4279)

* Updated config to fix CI test issue

Signed-off-by: Virginia Adams <[email protected]>

* Increased num workers

Signed-off-by: Virginia Adams <[email protected]>

* verbose k2 install, skip if failed (#4289)

Signed-off-by: Aleksandr Laptev <[email protected]>

Co-authored-by: Aleksandr Laptev <[email protected]>

* Changed total virtual prompt tokens (#4295)

* Changed total virtual prompt tokens

Signed-off-by: Virginia Adams <[email protected]>

* put number of workers back

Signed-off-by: Virginia Adams <[email protected]>

* upper bound lightning

Signed-off-by: ericharper <[email protected]>

* update branch

Signed-off-by: ericharper <[email protected]>

* update config

Signed-off-by: ericharper <[email protected]>

* remove duplicate test

Signed-off-by: ericharper <[email protected]>

* fix tn test cases

Signed-off-by: ericharper <[email protected]>

* add another safe.directory

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: PeganovAnton <[email protected]>
Co-authored-by: treacker <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Nithin Rao <[email protected]>
Co-authored-by: fayejf <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: bene-ges <[email protected]>
Co-authored-by: Alexandra Antonova <[email protected]>
Co-authored-by: ekmb <[email protected]>
Co-authored-by: Virginia Adams <[email protected]>
Co-authored-by: Ghasem <[email protected]>
Co-authored-by: Ghasem Pasandi <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* [TTS] Training Fastpitch on German text and phonemes and finetuning HiFi-GAN on predicted mels (#4266)

* initial commit

Signed-off-by: Akshit Arora <[email protected]>

* cleared notebook outputs

Signed-off-by: Akshit Arora <[email protected]>

* formatting errors

Signed-off-by: Akshit Arora <[email protected]>

* formatting

Signed-off-by: Akshit Arora <[email protected]>

* addressed comments

Signed-off-by: Akshit Arora <[email protected]>

* addressed comments on tutorial

Signed-off-by: Akshit Arora <[email protected]>

* updated tutorial

Signed-off-by: Akshit Arora <[email protected]>

* updated grammar and fastpitch description

Signed-off-by: Akshit Arora <[email protected]>

* updated with feedback

Signed-off-by: Akshit Arora <[email protected]>

* updated with feedback

Signed-off-by: Akshit Arora <[email protected]>

* removed unused imports

Signed-off-by: Akshit Arora <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Speedup the speech commands dataset processing script (#4347)

* Add multiprocessing support to the google speech commands dataset processing script

Signed-off-by: Shantanu Acharya <[email protected]>

* fix number of args error with __extract_all_files function

Signed-off-by: Shantanu Acharya <[email protected]>

* fix styling issues

Signed-off-by: Shantanu Acharya <[email protected]>

* fix bugs with silence set construction and update librosa output write to use soundfile write

Signed-off-by: Shantanu Acharya <[email protected]>

* add docstrings and return values in __construct_filepaths as dictionary

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* fix wrong requirement (#4349)

Signed-off-by: Yang Zhang <[email protected]>

Co-authored-by: Sandeep Subramanian <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Refactored path to manifest (#4251)

Signed-off-by: Evgeniy Shabalin <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* IPA support for TTS (#4310)

* IPA tokenizer and G2P untested draft

Signed-off-by: Jocelyn Huang <[email protected]>

* Add IPA CMUdict and new heteronyms list

Signed-off-by: Jocelyn Huang <[email protected]>

* Add draft FastPitch IPA config

Signed-off-by: Jocelyn Huang <[email protected]>

* Minor bugfixes for IPA training

Signed-off-by: Jocelyn Huang <[email protected]>

* Add phoneme_probability to IPA G2P

Signed-off-by: Jocelyn Huang <[email protected]>

* Updates to IPA FastPitch training config

Signed-off-by: Jocelyn Huang <[email protected]>

* Update IPA dict and heteronyms file

Signed-off-by: Jocelyn Huang <[email protected]>

* Adjust default lr for IPA FastPitch to 1e-3

Signed-off-by: Jocelyn Huang <[email protected]>

* Rename IPA CMUdict to reflect date

Signed-off-by: Jocelyn Huang <[email protected]>

* Add docstrings for IPA tokenizer and G2P, update CMUdict path for config

Signed-off-by: Jocelyn Huang <[email protected]>

* Fix IPA vocab ordering, add options to uppercase graphemes and remove stress symbols

Signed-off-by: Jocelyn Huang <[email protected]>

* Mark IPA classes as experimental

Signed-off-by: Jocelyn Huang <[email protected]>

* Update apostrophe-S cases

Signed-off-by: Jocelyn Huang <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Tn install (#4055)

* remove conda pynini requirement

Signed-off-by: Yang Zhang <[email protected]>

* remove remnants

Signed-off-by: Yang Zhang <[email protected]>

* merge with main

Signed-off-by: Yang Zhang <[email protected]>

* removing nlp collection dependency from text processing and thus breaking cyclyc imports

Signed-off-by: Yang Zhang <[email protected]>

* fix

Signed-off-by: Yang Zhang <[email protected]>

* fix wrong requirement

Signed-off-by: Yang Zhang <[email protected]>

* fix bug in vi

Signed-off-by: Yang Zhang <[email protected]>

* update jenkins folders

Signed-off-by: ekmb <[email protected]>

Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: ekmb <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* fix tutorial (#4352)

Signed-off-by: stevehuang52 <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* fix the post ln (#4350)

Signed-off-by: Yi Dong <[email protected]>

Co-authored-by: Eric Harper <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* [Fix] Hanging for Fully Randomized Bucketing (#4348)

* Update container to 22.05 (#4329)

* update container to 22.05

Signed-off-by: ericharper <[email protected]>

* try adding safe directory

Signed-off-by: ericharper <[email protected]>

* try env var

Signed-off-by: ericharper <[email protected]>

* printenv

Signed-off-by: ericharper <[email protected]>

* try GIT_BRANCH

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

* remove dbug statements

Signed-off-by: ericharper <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>

* Merge r1.9.0 main (#4331)

* update branch

Signed-off-by: ericharper <[email protected]>

* update package info

Signed-off-by: ericharper <[email protected]>

* cleaned up TN/ ITN doc (#4119)

* cleaned up TN/ ITN doc

Signed-off-by: Yang Zhang <[email protected]>

* fix typo

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* Draft: Fix restoring from checkpoint for case when `model.common_dataset_parameters.label_vocab_dir` is provided (#4136)

* Fix restoring from checkpoint with label vocab dir

Signed-off-by: PeganovAnton <[email protected]>

* Add tests for various ways to pass label ids to model

Signed-off-by: PeganovAnton <[email protected]>

* Fix typo

Signed-off-by: PeganovAnton <[email protected]>

* Fix typo

Signed-off-by: PeganovAnton <[email protected]>

* Do not create tmp directory

Signed-off-by: PeganovAnton <[email protected]>

* Fix parameter name

Signed-off-by: PeganovAnton <[email protected]>

* finish cherry-pick op

Signed-off-by: PeganovAnton <[email protected]>

* Fix labels errors

Signed-off-by: PeganovAnton <[email protected]>

* Remove duplicate stage

Signed-off-by: PeganovAnton <[email protected]>

* Change target branch

Signed-off-by: PeganovAnton <[email protected]>

* fix doc (#4146)

Signed-off-by: Yang Zhang <[email protected]>

* Tacotron2 retrain (#4103)

* fix yaml

Signed-off-by: treacker <[email protected]>

* Fix for new TTSDataset class

Signed-off-by: treacker <[email protected]>

* added wandb logging

Signed-off-by: treacker <[email protected]>

* added wandb logging

Signed-off-by: treacker <[email protected]>

* fix numpy version

Signed-off-by: treacker <[email protected]>

* fix numpy version

Signed-off-by: treacker <[email protected]>

* inference fix

Signed-off-by: treacker <[email protected]>

* removed old code

Signed-off-by: treacker <[email protected]>

* updated parser logic

Signed-off-by: treacker <[email protected]>

* reverted version update

Signed-off-by: treacker <[email protected]>

* refactored parser logic

Signed-off-by: treacker <[email protected]>

* Updated Jenkinsfile

Signed-off-by: treacker <[email protected]>

* Refactored tutorial for Tacotron2

Signed-off-by: treacker <[email protected]>

* Made backward compatibility

Signed-off-by: treacker <[email protected]>

* Made backward compatibility

Signed-off-by: treacker <[email protected]>

* Update Jenkinsfile

Signed-off-by: treacker <[email protected]>

* Update tacotron.yaml

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* cleaned up TN/ ITN doc (#4119)

* cleaned up TN/ ITN doc

Signed-off-by: Yang Zhang <[email protected]>

* fix typo

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>

* fix image

Signed-off-by: Yang Zhang <[email protected]>
Signed-off-by: treacker <[email protected]>

* Check implicit grad acc in GLUE dataset building (#4123)

* Check implicit grad acc in GLUE dataset building

Signed-off-by: MaximumEntropy <[email protected]>

* Fix jenkins test for GLUE/XNLI

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Fixed jenkins

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

* Refactoring

Signed-off-by: treacker <[email protected]>

Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>

* Multiprocess improvements (#4127)

* initial commit

Signed-off-by: nithinraok <[email protected]>

* start fix

Signed-off-by: nithinraok <[email protected]>

* improve multiprocessing speed while creating speaker dataset

Signed-off-by: nithinraok <[email protected]>

* updated scp to filelist

Signed-off-by: nithinraok <[email protected]>

* notebooks' link, typo and import  fix  (#4158)

* redo missing pr 4007

Signed-off-by: fayejf <[email protected]>

* remove extremely unreliable links

Signed-off-by: fayejf <[email protected]>

* update speaker docs (#4164)

* update speaker docs

Signed-off-by: nithinraok <[email protected]>

* chunks -> segments

Signed-off-by: nithinraok <[email protected]>

* Khz -> kHz

Signed-off-by: nithinraok <[email protected]>

* small fix (#4180)

Signed-off-by: fayejf <[email protected]>

* fix the server key value problem (#4196)

Signed-off-by: Yi Dong <[email protected]>

* Fix/punctuation/trainer required for setting test data (#4199)

* Draft of fix

Signed-off-by: PeganovAnton <[email protected]>

* Add warnings and replace globa_step with current_epoch

Signed-off-by: PeganovAnton <[email protected]>

* Small improvements to warnings

Signed-off-by: PeganovAnton <[email protected]>

* Error and warning messages improvements

Signed-off-by: PeganovAnton <[email protected]>

* Replace self.trainer with self._trainer

Signed-off-by: PeganovAnton <[email protected]>

* Update ContextNet version (#4207)

Signed-off-by: smajumdar <[email protected]>

* fix bugs for dialogue tutorial (#4211)

Signed-off-by: Zhilin Wang <[email protected]>

* Dialogue tutorial fix (#4214)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* Add docs for Thutmose Tagger (#4173)

* Add docs for Thutmose Tagger

Signed-off-by: Alexandra Antonova <[email protected]>

* add level in docs

Signed-off-by: Alexandra Antonova <[email protected]>

* delete folder to avoid error with running when folder exists from previous run

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>
Co-authored-by: ekmb <[email protected]>

* Dialogue tutorial fix (#4218)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously deleted files

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* Dialogue tutorial fix (#4221)

* fix bugs for dialogue tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* update path for convert_datasets.py due to conflict PR

Signed-off-by: Zhilin Wang <[email protected]>

* restore previously deleted files

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* update tutorial

Signed-off-by: Zhilin Wang <[email protected]>

* fix syntax error in ipynb-file (#4228)

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>

* fix json serialize (#4235)

Signed-off-by: Yi Dong <[email protected]>

* Prompt Learning Typo Fixes (#4238)

* Prompt tuning notebook typo fixes

Signed-off-by: Virginia Adams <[email protected]>

* Update tutorials.rst

* Update prompt_learning.rst

* Update prompt_learning.rst

* fixing bug 3642622 (#4250)

* fixing bug 3642622

Signed-off-by: Ghasem Pasandi <[email protected]>

* fixing bug 3642622

Signed-off-by: Ghasem Pasandi <[email protected]>

Co-authored-by: Ghasem Pasandi <[email protected]>

* fix broken link in the tutorial (#4257)

Signed-off-by: Alexandra Antonova <[email protected]>

Co-authored-by: Alexandra Antonova <[email protected]>

* Typo fix, branch change, better download messagae (#4262)

Signed-off-by: Virginia Adams <[email protected]>

* Raise error if bicleaner is not installed in NMT Data preprocesing notebook (#4264)

* Raise error if bicleaner is not installed

Signed-off-by: MaximumEntropy <[email protected]>

* Clear cells

Signed-off-by: MaximumEntropy <[email protected]>

* Fix missing validation dataset, whitelist certain keywords for datasets (#4269)

* Fix missing validation dataset, whitelist certain keywords for datasets

Signed-off-by: smajumdar <[email protected]>

* Fix missing validation dataset, whitelist certain keywords for datasets

Signed-off-by: smajumdar <[email protected]>

* Update asr configs with num_workers and pin_memory (#4270)

Signed-off-by: smajumdar <[email protected]>

* Fix epoch end (#4265)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* Set Save on train end to false (#4274)

* Set Save on train end to false

Signed-off-by: Virginia Adams <[email protected]>

* Update prompt_learning.rst

* Update prompt_learning.rst

* Update YAML (#4261)

Signed-off-by: MaximumEntropy <[email protected]>

* Updated config to fix CI test OOM error (#4279)

* Updated config to fix CI test issue

Signed-off-by: Virginia Adams <[email protected]>

* Increased num workers

Signed-off-by: Virginia Adams <[email protected]>

* verbose k2 install, skip if failed (#4289)

Signed-off-by: Aleksandr Laptev <[email protected]>

Co-authored-by: Aleksandr Laptev <[email protected]>

* Changed total virtual prompt tokens (#4295)

* Changed total virtual prompt tokens

Signed-off-by: Virginia Adams <[email protected]>

* put number of workers back

Signed-off-by: Virginia Adams <[email protected]>

* upper bound lightning

Signed-off-by: ericharper <[email protected]>

* update branch

Signed-off-by: ericharper <[email protected]>

* update config

Signed-off-by: ericharper <[email protected]>

* remove duplicate test

Signed-off-by: ericharper <[email protected]>

* fix tn test cases

Signed-off-by: ericharper <[email protected]>

* add another safe.directory

Signed-off-by: ericharper <[email protected]>

* typo

Signed-off-by: ericharper <[email protected]>

Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: PeganovAnton <[email protected]>
Co-authored-by: treacker <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Nithin Rao <[email protected]>
Co-authored-by: fayejf <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: bene-ges <[email protected]>
Co-authored-by: Alexandra Antonova <[email protected]>
Co-authored-by: ekmb <[email protected]>
Co-authored-by: Virginia Adams <[email protected]>
Co-authored-by: Ghasem <[email protected]>
Co-authored-by: Ghasem Pasandi <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>

* fix full_randn bucket hang

Signed-off-by: stevehuang52 <[email protected]>

* remove unused variables

Signed-off-by: stevehuang52 <[email protected]>

Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: PeganovAnton <[email protected]>
Co-authored-by: treacker <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Nithin Rao <[email protected]>
Co-authored-by: fayejf <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: bene-ges <[email protected]>
Co-authored-by: Alexandra Antonova <[email protected]>
Co-authored-by: ekmb <[email protected]>
Co-authored-by: Virginia Adams <[email protected]>
Co-authored-by: Ghasem <[email protected]>
Co-authored-by: Ghasem Pasandi <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Co-authored-by: Aleksandr Laptev <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Bits of RADTTS support (#4343)

* Bits of RADTTS support

Signed-off-by: Boris Fomitchev <[email protected]>

* Fixed args mismatch

Signed-off-by: Boris Fomitchev <[email protected]>

* Style

Signed-off-by: Boris Fomitchev <[email protected]>

* Addressed review comments

Signed-off-by: Boris Fomitchev <[email protected]>

* More review comments

Signed-off-by: Boris Fomitchev <[email protected]>

Co-authored-by: Eric Harper <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Prompt Learning Pipeline Parallel (#4291)

* Added get_forward_output_and_loss_func and updated train/val steps

Signed-off-by: Virginia Adams <[email protected]>

* Added preprocess flag before prompt table/encoder access

Signed-off-by: Virginia Adams <[email protected]>

* Made two optimizer groups, one for frozen, one for soft prompt

Signed-off-by: Virginia Adams <[email protected]>

* Pipeline parallel working

Signed-off-by: Virginia Adams <[email protected]>

* Still figuring out setting lr/sched for one param group

Signed-off-by: Virginia Adams <[email protected]>

* Set betas to zero

Signed-off-by: Virginia Adams <[email protected]>

* Only unfreeze one sublayer with lr 0.0

Signed-off-by: Virginia Adams <[email protected]>

* Pipeline parallel working w/ one optimizer

Signed-off-by: Virginia Adams <[email protected]>

* Trying to fix Jenkins file

Signed-off-by: Virginia Adams <[email protected]>

* Trying to fix Jenkins file

Signed-off-by: Virginia Adams <[email protected]>

* Getting updated jenkins test to work

Signed-off-by: Virginia Adams <[email protected]>

* Getting updated jenkins test to work

Signed-off-by: Virginia Adams <[email protected]>

* added prompt learning tp and pp CI tests

Signed-off-by: Virginia Adams <[email protected]>

* Added amp_o2 model compatibility

Signed-off-by: Virginia Adams <[email protected]>

* Made CI test smaller

Signed-off-by: Virginia Adams <[email protected]>

* Still trying to get Jenkins to work

Signed-off-by: Virginia Adams <[email protected]>

* Still trying to get Jenkins to work

Signed-off-by: Virginia Adams <[email protected]>

* Temporarily moving prompt learning CI test to beginning

Signed-off-by: Virginia Adams <[email protected]>

* Changing the layer being unfrozen

Signed-off-by: Virginia Adams <[email protected]>

* debug jenkins

Signed-off-by: Virginia Adams <[email protected]>

* Move pp unfreeze to init

Signed-off-by: Virginia Adams <[email protected]>

* Try to make Jenkins test parallel

Signed-off-by: Virginia Adams <[email protected]>

* Fix python formatting

Signed-off-by: Virginia Adams <[email protected]>

* Moved prompt learning tests back to where they belong

Signed-off-by: Virginia Adams <[email protected]>

* add back checkpoint convertion CI test

Signed-off-by: Virginia Adams <[email protected]>

* Revert "add back checkpoint convertion CI test"

This reverts commit 61e2ffcdefe964c8e74b74d8c10906ae29f32b6d.

* Add back checkpoint conversion test

Signed-off-by: Virginia Adams <[email protected]>

* Setting requires grad to True everywhere

Signed-off-by: Virginia Adams <[email protected]>

* Updated config comments and simplified param group code

Signed-off-by: Virginia Adams <[email protected]>

* Added comment on frozen_model having lr=0.0

Signed-off-by: Virginia Adams <[email protected]>

* Added configure optimizers methods

Signed-off-by: Virginia Adams <[email protected]>

* Set amp_o2 to false

Signed-off-by: Virginia Adams <[email protected]>

* removed o2 code

Signed-off-by: Virginia Adams <[email protected]>

* Python formatting fix

Signed-off-by: Virginia Adams <[email protected]>

Co-authored-by: Eric Harper <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* auto switch conformer encoder adapter in_features (#4354)

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Dataloader, collector, loss and metric for multiscale diarization decoder  (#4187)

* First commit

Signed-off-by: Taejin Park <[email protected]>

* Checked funtionality and imports

Signed-off-by: Taejin Park <[email protected]>

* fixed import issues

Signed-off-by: Taejin Park <[email protected]>

* Removed the changed made by mistake

Signed-off-by: Taejin Park <[email protected]>

* Style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM errors 001

Signed-off-by: Taejin Park <[email protected]>

* Fixed LGTM and style fix

Signed-off-by: Taejin Park <[email protected]>

* Changed docstrings

Signed-off-by: Taejin Park <[email protected]>

* LGTM again

Signed-off-by: Taejin Park <[email protected]>

* Removed unnecessary torch setting lines

Signed-off-by: Taejin Park <[email protected]>

* Style fix and isort

Signed-off-by: Taejin Park <[email protected]>

* jbalam-nv comments reflected

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected comments and created _diar_label.py

Signed-off-by: Taejin Park <[email protected]>

* Typo fix and style fix

Signed-off-by: Taejin Park <[email protected]>

* Fixed target_spks[0] index error

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* LGTM unused import IterDataset

Signed-off-by: Taejin Park <[email protected]>

* revert collection doc year

Signed-off-by: Taejin Park <[email protected]>

* Code format error in collections.py

Signed-off-by: Taejin Park <[email protected]>

* fix collections space format error

Signed-off-by: Taejin Park <[email protected]>

* merged main correctly

Signed-off-by: Taejin Park <[email protected]>

* style fix

Signed-off-by: Taejin Park <[email protected]>

* Reflected all comments and tested

Signed-off-by: Taejin Park <[email protected]>

* style fix and LGTM

Signed-off-by: Taejin Park <[email protected]>

* rttm_filepath to rttm_file and removed self included funcs, tested

Signed-off-by: Taejin Park <[email protected]>

Co-authored-by: Nithin Rao <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Add ASR CTC Decoding module (#4342)

* Initial commit

Signed-off-by: smajumdar <[email protected]>

* Full support for decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Temp

Signed-off-by: smajumdar <[email protected]>

* Fix labels of y_sequence

Signed-off-by: smajumdar <[email protected]>

* Set support for sentencepiece subword merging

Signed-off-by: smajumdar <[email protected]>

* Fix char and word based token merge alignment

Signed-off-by: smajumdar <[email protected]>

* Revert incorrect change

Signed-off-by: smajumdar <[email protected]>

* Update docstring

Signed-off-by: smajumdar <[email protected]>

* Improve compatibility with greedy tokens and log probs

Signed-off-by: smajumdar <[email protected]>

* Update scripts to use decoding strategy

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Add tests and docs

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix speaker decoder timestamps

Signed-off-by: smajumdar <[email protected]>

* Fix decoding of ctc models

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>

* Address reviewer comments

Signed-off-by: smajumdar <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Option to disable mp in VAD via num_workers=1 (#4317)

* Option to disable mp in VAD via num_workers=1

In certain environments python multiprocessing can deadlock. This adds a convenient version to disable by setting num_workers to 1.

Signed-off-by: Georg Kucsko <[email protected]>

* add none handling

Signed-off-by: Georg Kucsko <[email protected]>

* additional none handling

Signed-off-by: Georg Kucsko <[email protected]>

Co-authored-by: fayejf <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* remove redundant bias expand (#4382)

* remove redundant bias expand

Signed-off-by: Xiaowei Ren <[email protected]>

* delete redundant code

Signed-off-by: Xiaowei Ren <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Add option for specifying wandb save_dir from config (#4379)

* give option to user to specify wandb save dir via config

Signed-off-by: Shantanu Acharya <[email protected]>

* create save_dir directory for wandb logger if not exists

Signed-off-by: Shantanu Acharya <[email protected]>

* update save_dir get method with a default value

Signed-off-by: Shantanu Acharya <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (#4383)

Signed-off-by: tbartley94 <[email protected]>

Co-authored-by: tbartley94 <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* [Bugfix][TTS] wrong order of returned tuple for general_collate_fn. (#4388)

Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>

* Merge r1.10.0 main (#4398)

* update branch

Signed-off-by: ericharper <[email protected]>

* Set headscale false (#4364)

Signed-off-by: MaximumEntropy <[email protected]>

* Add wandb as dependency (#4365)

Signed-off-by: smajumdar <[email protected]>

* Raise trainer error (#4356)

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Set headscale false (#4364) (#4366)

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: smajumdar <[email protected]>

* Finetuning changes for BART (#4003)

* Temp

Signed-off-by: MaximumEntropy <[email protected]>

* Checkpoint converter to nemo for bart

Signed-off-by: MaximumEntropy <[email protected]>

* Style

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Make position embedding expansion specific to a batch to avoid checkpoint size mismatches (#4357)

* Style

Signed-off-by: MaximumEntropy <[email protected]>

* Fix logging warning

Signed-off-by: MaximumEntropy <[email protected]>

Co-authored-by: Micha Livne <[email protected]>

* Fix electronic bug, new time ITN rule (#4355)

* fix electronic bug

Signed-off-by: ekmb <[email protected]>

* add new itn time rule

Signed-off-by: ekmb <[email protected]>

* revert domain changes

Signed-off-by: ekmb <[email protected]>

* remove repetition

Signed-off-by: ekmb <[email protected]>

* Correct support for dataclasses in default module dim (#4372)

* Correct support for dataclasses in default module dim

Signed-off-by: smajumdar <[email protected]>

* Fix path for save of results

Signed-off-by: smajumdar <[email protected]>

* fix pad id bug (#4377)

Signed-off-by: Yi Dong <[email protected]>

* Question answering bug fix (#4381)

* refactor dialogue state tracking for modelling/dataset interoperability

Signed-off-by: Zhilin Wang <[email protected]>

* fix style changes

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* fix style raised by lgtm

Signed-off-by: Zhilin Wang <[email protected]>

* fix style formatting

Signed-off-by: Zhilin Wang <[email protected]>

* update template to include description of intent

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* changes based on requests in review

Signed-off-by: Zhilin Wang <[email protected]>

* add compatibility with assistant dataset

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins

Signed-off-by: Zhilin Wang <[email protected]>

* remove dialogue_state_tracking

Signed-off-by: Zhilin Wang <[email protected]>

* update huggingface utils for dialogue

Signed-off-by: Zhilin Wang <[email protected]>

* rename dialogue_state_tracking_hybrid to dialogue_state_tracking_sgdqa

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

Signed-off-by: Zhilin Wang <[email protected]>

* fix style

Signed-off-by: Zhilin Wang <[email protected]>

* style fix nemo/collections/nlp/models/dialogue_state_tracking_sgdqa/__init__.py

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkinsfile for SGDGEN

Signed-off-by: Zhilin Wang <[email protected]>

* fix typo

Signed-off-by: Zhilin Wang <[email protected]>

* add docstrings for assistant data processsor

Signed-off-by: Zhilin Wang <[email protected]>

* update Jenkins for SGDGEN local checkpoint

Signed-off-by: Zhilin Wang <[email protected]>

* update style

Signed-off-by: Zhilin Wang <[email protected]>

* use local vocab file for Jenkinsfile

Signed-off-by: Zhilin Wang <[email protected]>

* patch for Jenkins CI using local file

Signed-off-by: Zhilin Wang <[email protected]>

* add slot filling prediction and metrics

Signed-off-by: Zhilin Wang <[email protected]>

* remove unused code

Signed-off-by: Zhilin Wang <[email protected]>

* style fix

…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants