[Codec] Update codec checkpoint config

[anteju]

What does this PR do ?

Update codec configs to always save NeMo checkpoint with best val_loss.

Collection: TTS

Changelog

• Updated checkpoint_callback_params to always save NeMo checkpoint with best val_loss and to save 5 best checkpoints
• Set create_wandb_logger to true, since model.log_config.log_wandb is set to true by default

Usage

N/A

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you add or update any necessary documentation?
• Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

• New Feature
• Bugfix
• Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

• Related to # (issue)

[rlangman]

Should we keep the convention that all ??? be at the top of the file? I think TTS is the only domain which does it currently.

[anteju]

I don't have a strong preference.
We could move to the top, set it to project: audio_codec or possibly remove altogether (wandb.init API indicates it's optional, but haven't tested it).

[rlangman]

I think for wandb the "name" field is supposed to be the name of a specific run to track. "project" is the category to sort runs by.

So project could be set to ${name} or some other hardcoded value. Name can be removed, assuming user will override it if they want to organize their runs (if not provided then wandb generates a random name like happy-giraffe-42)

[anteju]

We can set both to null, seems to be common across other recipes:

  wandb_logger_kwargs:
    name: null
    project: null

[anteju]

jenkins

[anteju]

CI failing with an unrelated error

FAILED tests/core/test_save_restore.py::TestSaveRestore::test_hf_model_info_with_card_data - AssertionError: assert not True

 +  where True = hasattr(ModelInfo(id='Koleshjr/stt_sw_40k_10k_3e_4_5_epoch', author='Koleshjr', sha='bb602f31a8bdd466af474b8bfb735ebf26bded51', last_modified=datetime.datetime(2023, 11, 8, 12, 41, 13, tzinfo=datetime.timezone.utc), private=False, gated=None, disabled=None, downloads=92, likes=0, library_name='nemo', tags=['nemo', 'region:us'], pipeline_tag=None, mask_token=None, card_data=None, widget_data=None, model_index=None, config=None, transformers_info=None, siblings=[RepoSibling(rfilename='.gitattributes', size=None, blob_id=None, lfs=None), RepoSibling(rfilename='readme_template.md', size=None, blob_id=None, lfs=None), RepoSibling(rfilename='stt_sw_40k_10k_3e_4_5_epoch.nemo', size=None, blob_id=None, lfs=None)], spaces=None, safetensors=None), 'cardData')

= 1 failed, 1562 passed, 267 skipped, 22 deselected, 530 warnings in 834.39s (0:13:54) =

script returned exit code 1

[nithinraok]

Thanks Ante. LGTM.

[anteju]

CI failing on test-nlp-imports, seems unrelated and other PRs are failing atm.

[nithinraok]

They are coming from megatron.
ModuleNotFoundError: No module named 'transformer_engine

[anteju]

Rebasing onto main to hopefully resolve CI failure.

[anteju]

jenkins