Fix bug in RNNT Joint WER calculation for fused batch #8587
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: smajumdar <[email protected]>
|
jenkins |
pzelasko
pushed a commit
that referenced
this pull request
Mar 7, 2024
Signed-off-by: smajumdar <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]>
zpx01
pushed a commit
to zpx01/NeMo
that referenced
this pull request
Mar 8, 2024
Signed-off-by: smajumdar <[email protected]> Signed-off-by: Zeeshan Patel <[email protected]>
Agoniii
pushed a commit
to Agoniii/NeMo
that referenced
this pull request
Mar 15, 2024
Signed-off-by: smajumdar <[email protected]> Signed-off-by: Agoniii <[email protected]>
JRD971000
pushed a commit
that referenced
this pull request
Mar 15, 2024
Signed-off-by: smajumdar <[email protected]> Signed-off-by: ataghibakhsh <[email protected]>
pablo-garay
pushed a commit
that referenced
this pull request
Mar 19, 2024
Signed-off-by: smajumdar <[email protected]> Signed-off-by: Pablo Garay <[email protected]>
pzelasko
added a commit
that referenced
this pull request
Apr 16, 2024
…t-only) dataloading (#8581) * wip Signed-off-by: Piotr Żelasko <[email protected]> * Partially working config groups Signed-off-by: Piotr Żelasko <[email protected]> * Working test with abasic group in the input config Signed-off-by: Piotr Żelasko <[email protected]> * Working test with nested groups in input config Signed-off-by: Piotr Żelasko <[email protected]> * Working test with specifying a YAML path for input_cfg Signed-off-by: Piotr Żelasko <[email protected]> * a very rough example of text dataloading via lhotse Signed-off-by: Piotr Żelasko <[email protected]> * Cleaner integration of multimodal audio/text loading that allows to control the effective audio vs text size (requires latest lhotse) Signed-off-by: Piotr Żelasko <[email protected]> * remove obsolete test Signed-off-by: Piotr Żelasko <[email protected]> * Fix an import in export_utils.py (#8571) Signed-off-by: w4-jinhyeonkim <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Yttm deprecation (#8322) * yttm deprecation init commit Signed-off-by: AlexGrinch <[email protected]> * removed tests Signed-off-by: AlexGrinch <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * bug fix Signed-off-by: AlexGrinch <[email protected]> * path fix Signed-off-by: AlexGrinch <[email protected]> * fixing path Signed-off-by: AlexGrinch <[email protected]> * updated tests to spm Signed-off-by: AlexGrinch <[email protected]> * updated Jenkinsfile Signed-off-by: AlexGrinch <[email protected]> * new model with spm in tests Signed-off-by: AlexGrinch <[email protected]> * yttm removed Signed-off-by: AlexGrinch <[email protected]> * updated aayn config Signed-off-by: AlexGrinch <[email protected]> --------- Signed-off-by: AlexGrinch <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Piotr Żelasko <[email protected]> * Fixed missing copy import in rnnt_decoder.py (#8580) * Added copy import to rnnt_decoding.py Signed-off-by: Isaac McFadyen <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Isaac McFadyen <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Piotr Żelasko <[email protected]> * Fix bug in RNNT Joint WER calculation for fused batch (#8587) Signed-off-by: smajumdar <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Fixed Context Parallel HtoD sync (#8557) * Fixed cp HtoD sync Signed-off-by: Selvaraj Anandaraj <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Piotr Żelasko <[email protected]> * change default and add key to config files (#8594) Signed-off-by: Chen Cui <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Fix triton import guards (#8552) * Fix triton import guards Signed-off-by: Michal Futrega <[email protected]> * Update attention.py Signed-off-by: Michal Futrega <[email protected]> --------- Signed-off-by: Michal Futrega <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Add config key for dropout position in LoRA adapter (#8583) Signed-off-by: Michal Futrega <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * fix ia3 mlp infused adapter (#8597) Signed-off-by: Chen Cui <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Prevent Redundant Gather for LoRA Sequence Parallel (#8602) * enable layernorm output gathered Signed-off-by: Chen Cui <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Piotr Żelasko <[email protected]> * Accelerate `transcribe_speech.py` for short-form data: pre-sorting support (#8564) * POC using bucketing in transcribe_speech.py Signed-off-by: Piotr Żelasko <[email protected]> * extend to multi task aed Signed-off-by: Piotr Żelasko <[email protected]> * fixes for aed multi task text/lang field selectors Signed-off-by: Piotr Żelasko <[email protected]> * remove assert Signed-off-by: Piotr Żelasko <[email protected]> * fix Signed-off-by: Piotr Żelasko <[email protected]> * expose option for bucket buffer size Signed-off-by: Piotr Żelasko <[email protected]> * fixes, ctc support Signed-off-by: Piotr Żelasko <[email protected]> * support pre-sorting manifests in transcribe_speech.py Signed-off-by: Piotr Żelasko <[email protected]> * cleanup Signed-off-by: Piotr Żelasko <[email protected]> * reorder transcriptions back to original manifest order Signed-off-by: Piotr Żelasko <[email protected]> * remove bucketing entirely Signed-off-by: Piotr Żelasko <[email protected]> * code review changes Signed-off-by: Piotr Żelasko <[email protected]> * code review changes--amend Signed-off-by: Piotr Żelasko <[email protected]> * refactor text_field/lang_field passing Signed-off-by: Piotr Żelasko <[email protected]> * Fix reordering bug; disable presorting for multi task for now Signed-off-by: Piotr Żelasko <[email protected]> * Add support for presort + multi task model Signed-off-by: Piotr Żelasko <[email protected]> * Code reviews Signed-off-by: Piotr Żelasko <[email protected]> * Fix jenkins tests, add user-friendly error msg for canary Signed-off-by: Piotr Żelasko <[email protected]> --------- Signed-off-by: Piotr Żelasko <[email protected]> * fix tests Signed-off-by: Piotr Żelasko <[email protected]> * Bump min required lhotse version Signed-off-by: Piotr Żelasko <[email protected]> * Add some documentation about this config format and the multimodal features Signed-off-by: Piotr Żelasko <[email protected]> * Add caution about multiple shards Signed-off-by: Piotr Żelasko <[email protected]> * Address Tom's code review Signed-off-by: Piotr Żelasko <[email protected]> * Add copyright header Signed-off-by: Piotr Żelasko <[email protected]> * Fix (hopefully) issue with forced ascii encoding in CI Signed-off-by: Piotr Żelasko <[email protected]> * Support resolving input_cfg path into config contents Signed-off-by: Piotr Żelasko <[email protected]> * Code review changes in docs Signed-off-by: Piotr Żelasko <[email protected]> * Fix unicode decode error Signed-off-by: Piotr Żelasko <[email protected]> --------- Signed-off-by: Piotr Żelasko <[email protected]> Signed-off-by: w4-jinhyeonkim <[email protected]> Signed-off-by: AlexGrinch <[email protected]> Signed-off-by: Isaac McFadyen <[email protected]> Signed-off-by: smajumdar <[email protected]> Signed-off-by: Selvaraj Anandaraj <[email protected]> Signed-off-by: Chen Cui <[email protected]> Signed-off-by: Michal Futrega <[email protected]> Co-authored-by: w4-jinhyeonkim <[email protected]> Co-authored-by: Aleksey Grinchuk (Oleksii Hrinchuk) <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Isaac McFadyen <[email protected]> Co-authored-by: Somshubra Majumdar <[email protected]> Co-authored-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: Chen Cui <[email protected]> Co-authored-by: Michal Futrega <[email protected]> Co-authored-by: Pablo Garay <[email protected]>
xingyaoww
pushed a commit
to xingyaoww/NeMo
that referenced
this pull request
Apr 23, 2024
…t-only) dataloading (NVIDIA#8581) * wip Signed-off-by: Piotr Żelasko <[email protected]> * Partially working config groups Signed-off-by: Piotr Żelasko <[email protected]> * Working test with abasic group in the input config Signed-off-by: Piotr Żelasko <[email protected]> * Working test with nested groups in input config Signed-off-by: Piotr Żelasko <[email protected]> * Working test with specifying a YAML path for input_cfg Signed-off-by: Piotr Żelasko <[email protected]> * a very rough example of text dataloading via lhotse Signed-off-by: Piotr Żelasko <[email protected]> * Cleaner integration of multimodal audio/text loading that allows to control the effective audio vs text size (requires latest lhotse) Signed-off-by: Piotr Żelasko <[email protected]> * remove obsolete test Signed-off-by: Piotr Żelasko <[email protected]> * Fix an import in export_utils.py (NVIDIA#8571) Signed-off-by: w4-jinhyeonkim <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Yttm deprecation (NVIDIA#8322) * yttm deprecation init commit Signed-off-by: AlexGrinch <[email protected]> * removed tests Signed-off-by: AlexGrinch <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * bug fix Signed-off-by: AlexGrinch <[email protected]> * path fix Signed-off-by: AlexGrinch <[email protected]> * fixing path Signed-off-by: AlexGrinch <[email protected]> * updated tests to spm Signed-off-by: AlexGrinch <[email protected]> * updated Jenkinsfile Signed-off-by: AlexGrinch <[email protected]> * new model with spm in tests Signed-off-by: AlexGrinch <[email protected]> * yttm removed Signed-off-by: AlexGrinch <[email protected]> * updated aayn config Signed-off-by: AlexGrinch <[email protected]> --------- Signed-off-by: AlexGrinch <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Piotr Żelasko <[email protected]> * Fixed missing copy import in rnnt_decoder.py (NVIDIA#8580) * Added copy import to rnnt_decoding.py Signed-off-by: Isaac McFadyen <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Isaac McFadyen <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Piotr Żelasko <[email protected]> * Fix bug in RNNT Joint WER calculation for fused batch (NVIDIA#8587) Signed-off-by: smajumdar <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Fixed Context Parallel HtoD sync (NVIDIA#8557) * Fixed cp HtoD sync Signed-off-by: Selvaraj Anandaraj <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Piotr Żelasko <[email protected]> * change default and add key to config files (NVIDIA#8594) Signed-off-by: Chen Cui <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Fix triton import guards (NVIDIA#8552) * Fix triton import guards Signed-off-by: Michal Futrega <[email protected]> * Update attention.py Signed-off-by: Michal Futrega <[email protected]> --------- Signed-off-by: Michal Futrega <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Add config key for dropout position in LoRA adapter (NVIDIA#8583) Signed-off-by: Michal Futrega <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * fix ia3 mlp infused adapter (NVIDIA#8597) Signed-off-by: Chen Cui <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Prevent Redundant Gather for LoRA Sequence Parallel (NVIDIA#8602) * enable layernorm output gathered Signed-off-by: Chen Cui <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Piotr Żelasko <[email protected]> * Accelerate `transcribe_speech.py` for short-form data: pre-sorting support (NVIDIA#8564) * POC using bucketing in transcribe_speech.py Signed-off-by: Piotr Żelasko <[email protected]> * extend to multi task aed Signed-off-by: Piotr Żelasko <[email protected]> * fixes for aed multi task text/lang field selectors Signed-off-by: Piotr Żelasko <[email protected]> * remove assert Signed-off-by: Piotr Żelasko <[email protected]> * fix Signed-off-by: Piotr Żelasko <[email protected]> * expose option for bucket buffer size Signed-off-by: Piotr Żelasko <[email protected]> * fixes, ctc support Signed-off-by: Piotr Żelasko <[email protected]> * support pre-sorting manifests in transcribe_speech.py Signed-off-by: Piotr Żelasko <[email protected]> * cleanup Signed-off-by: Piotr Żelasko <[email protected]> * reorder transcriptions back to original manifest order Signed-off-by: Piotr Żelasko <[email protected]> * remove bucketing entirely Signed-off-by: Piotr Żelasko <[email protected]> * code review changes Signed-off-by: Piotr Żelasko <[email protected]> * code review changes--amend Signed-off-by: Piotr Żelasko <[email protected]> * refactor text_field/lang_field passing Signed-off-by: Piotr Żelasko <[email protected]> * Fix reordering bug; disable presorting for multi task for now Signed-off-by: Piotr Żelasko <[email protected]> * Add support for presort + multi task model Signed-off-by: Piotr Żelasko <[email protected]> * Code reviews Signed-off-by: Piotr Żelasko <[email protected]> * Fix jenkins tests, add user-friendly error msg for canary Signed-off-by: Piotr Żelasko <[email protected]> --------- Signed-off-by: Piotr Żelasko <[email protected]> * fix tests Signed-off-by: Piotr Żelasko <[email protected]> * Bump min required lhotse version Signed-off-by: Piotr Żelasko <[email protected]> * Add some documentation about this config format and the multimodal features Signed-off-by: Piotr Żelasko <[email protected]> * Add caution about multiple shards Signed-off-by: Piotr Żelasko <[email protected]> * Address Tom's code review Signed-off-by: Piotr Żelasko <[email protected]> * Add copyright header Signed-off-by: Piotr Żelasko <[email protected]> * Fix (hopefully) issue with forced ascii encoding in CI Signed-off-by: Piotr Żelasko <[email protected]> * Support resolving input_cfg path into config contents Signed-off-by: Piotr Żelasko <[email protected]> * Code review changes in docs Signed-off-by: Piotr Żelasko <[email protected]> * Fix unicode decode error Signed-off-by: Piotr Żelasko <[email protected]> --------- Signed-off-by: Piotr Żelasko <[email protected]> Signed-off-by: w4-jinhyeonkim <[email protected]> Signed-off-by: AlexGrinch <[email protected]> Signed-off-by: Isaac McFadyen <[email protected]> Signed-off-by: smajumdar <[email protected]> Signed-off-by: Selvaraj Anandaraj <[email protected]> Signed-off-by: Chen Cui <[email protected]> Signed-off-by: Michal Futrega <[email protected]> Co-authored-by: w4-jinhyeonkim <[email protected]> Co-authored-by: Aleksey Grinchuk (Oleksii Hrinchuk) <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Isaac McFadyen <[email protected]> Co-authored-by: Somshubra Majumdar <[email protected]> Co-authored-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: Chen Cui <[email protected]> Co-authored-by: Michal Futrega <[email protected]> Co-authored-by: Pablo Garay <[email protected]>
alxzhang-amazon
pushed a commit
to alxzhang-amazon/NeMo
that referenced
this pull request
Apr 26, 2024
…t-only) dataloading (NVIDIA#8581) * wip Signed-off-by: Piotr Żelasko <[email protected]> * Partially working config groups Signed-off-by: Piotr Żelasko <[email protected]> * Working test with abasic group in the input config Signed-off-by: Piotr Żelasko <[email protected]> * Working test with nested groups in input config Signed-off-by: Piotr Żelasko <[email protected]> * Working test with specifying a YAML path for input_cfg Signed-off-by: Piotr Żelasko <[email protected]> * a very rough example of text dataloading via lhotse Signed-off-by: Piotr Żelasko <[email protected]> * Cleaner integration of multimodal audio/text loading that allows to control the effective audio vs text size (requires latest lhotse) Signed-off-by: Piotr Żelasko <[email protected]> * remove obsolete test Signed-off-by: Piotr Żelasko <[email protected]> * Fix an import in export_utils.py (NVIDIA#8571) Signed-off-by: w4-jinhyeonkim <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Yttm deprecation (NVIDIA#8322) * yttm deprecation init commit Signed-off-by: AlexGrinch <[email protected]> * removed tests Signed-off-by: AlexGrinch <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * bug fix Signed-off-by: AlexGrinch <[email protected]> * path fix Signed-off-by: AlexGrinch <[email protected]> * fixing path Signed-off-by: AlexGrinch <[email protected]> * updated tests to spm Signed-off-by: AlexGrinch <[email protected]> * updated Jenkinsfile Signed-off-by: AlexGrinch <[email protected]> * new model with spm in tests Signed-off-by: AlexGrinch <[email protected]> * yttm removed Signed-off-by: AlexGrinch <[email protected]> * updated aayn config Signed-off-by: AlexGrinch <[email protected]> --------- Signed-off-by: AlexGrinch <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Piotr Żelasko <[email protected]> * Fixed missing copy import in rnnt_decoder.py (NVIDIA#8580) * Added copy import to rnnt_decoding.py Signed-off-by: Isaac McFadyen <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Isaac McFadyen <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Piotr Żelasko <[email protected]> * Fix bug in RNNT Joint WER calculation for fused batch (NVIDIA#8587) Signed-off-by: smajumdar <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Fixed Context Parallel HtoD sync (NVIDIA#8557) * Fixed cp HtoD sync Signed-off-by: Selvaraj Anandaraj <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Piotr Żelasko <[email protected]> * change default and add key to config files (NVIDIA#8594) Signed-off-by: Chen Cui <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Fix triton import guards (NVIDIA#8552) * Fix triton import guards Signed-off-by: Michal Futrega <[email protected]> * Update attention.py Signed-off-by: Michal Futrega <[email protected]> --------- Signed-off-by: Michal Futrega <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Add config key for dropout position in LoRA adapter (NVIDIA#8583) Signed-off-by: Michal Futrega <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * fix ia3 mlp infused adapter (NVIDIA#8597) Signed-off-by: Chen Cui <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Prevent Redundant Gather for LoRA Sequence Parallel (NVIDIA#8602) * enable layernorm output gathered Signed-off-by: Chen Cui <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Piotr Żelasko <[email protected]> * Accelerate `transcribe_speech.py` for short-form data: pre-sorting support (NVIDIA#8564) * POC using bucketing in transcribe_speech.py Signed-off-by: Piotr Żelasko <[email protected]> * extend to multi task aed Signed-off-by: Piotr Żelasko <[email protected]> * fixes for aed multi task text/lang field selectors Signed-off-by: Piotr Żelasko <[email protected]> * remove assert Signed-off-by: Piotr Żelasko <[email protected]> * fix Signed-off-by: Piotr Żelasko <[email protected]> * expose option for bucket buffer size Signed-off-by: Piotr Żelasko <[email protected]> * fixes, ctc support Signed-off-by: Piotr Żelasko <[email protected]> * support pre-sorting manifests in transcribe_speech.py Signed-off-by: Piotr Żelasko <[email protected]> * cleanup Signed-off-by: Piotr Żelasko <[email protected]> * reorder transcriptions back to original manifest order Signed-off-by: Piotr Żelasko <[email protected]> * remove bucketing entirely Signed-off-by: Piotr Żelasko <[email protected]> * code review changes Signed-off-by: Piotr Żelasko <[email protected]> * code review changes--amend Signed-off-by: Piotr Żelasko <[email protected]> * refactor text_field/lang_field passing Signed-off-by: Piotr Żelasko <[email protected]> * Fix reordering bug; disable presorting for multi task for now Signed-off-by: Piotr Żelasko <[email protected]> * Add support for presort + multi task model Signed-off-by: Piotr Żelasko <[email protected]> * Code reviews Signed-off-by: Piotr Żelasko <[email protected]> * Fix jenkins tests, add user-friendly error msg for canary Signed-off-by: Piotr Żelasko <[email protected]> --------- Signed-off-by: Piotr Żelasko <[email protected]> * fix tests Signed-off-by: Piotr Żelasko <[email protected]> * Bump min required lhotse version Signed-off-by: Piotr Żelasko <[email protected]> * Add some documentation about this config format and the multimodal features Signed-off-by: Piotr Żelasko <[email protected]> * Add caution about multiple shards Signed-off-by: Piotr Żelasko <[email protected]> * Address Tom's code review Signed-off-by: Piotr Żelasko <[email protected]> * Add copyright header Signed-off-by: Piotr Żelasko <[email protected]> * Fix (hopefully) issue with forced ascii encoding in CI Signed-off-by: Piotr Żelasko <[email protected]> * Support resolving input_cfg path into config contents Signed-off-by: Piotr Żelasko <[email protected]> * Code review changes in docs Signed-off-by: Piotr Żelasko <[email protected]> * Fix unicode decode error Signed-off-by: Piotr Żelasko <[email protected]> --------- Signed-off-by: Piotr Żelasko <[email protected]> Signed-off-by: w4-jinhyeonkim <[email protected]> Signed-off-by: AlexGrinch <[email protected]> Signed-off-by: Isaac McFadyen <[email protected]> Signed-off-by: smajumdar <[email protected]> Signed-off-by: Selvaraj Anandaraj <[email protected]> Signed-off-by: Chen Cui <[email protected]> Signed-off-by: Michal Futrega <[email protected]> Co-authored-by: w4-jinhyeonkim <[email protected]> Co-authored-by: Aleksey Grinchuk (Oleksii Hrinchuk) <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Isaac McFadyen <[email protected]> Co-authored-by: Somshubra Majumdar <[email protected]> Co-authored-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: Chen Cui <[email protected]> Co-authored-by: Michal Futrega <[email protected]> Co-authored-by: Pablo Garay <[email protected]>
suiyoubi
pushed a commit
that referenced
this pull request
May 2, 2024
…t-only) dataloading (#8581) * wip Signed-off-by: Piotr Żelasko <[email protected]> * Partially working config groups Signed-off-by: Piotr Żelasko <[email protected]> * Working test with abasic group in the input config Signed-off-by: Piotr Żelasko <[email protected]> * Working test with nested groups in input config Signed-off-by: Piotr Żelasko <[email protected]> * Working test with specifying a YAML path for input_cfg Signed-off-by: Piotr Żelasko <[email protected]> * a very rough example of text dataloading via lhotse Signed-off-by: Piotr Żelasko <[email protected]> * Cleaner integration of multimodal audio/text loading that allows to control the effective audio vs text size (requires latest lhotse) Signed-off-by: Piotr Żelasko <[email protected]> * remove obsolete test Signed-off-by: Piotr Żelasko <[email protected]> * Fix an import in export_utils.py (#8571) Signed-off-by: w4-jinhyeonkim <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Yttm deprecation (#8322) * yttm deprecation init commit Signed-off-by: AlexGrinch <[email protected]> * removed tests Signed-off-by: AlexGrinch <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * bug fix Signed-off-by: AlexGrinch <[email protected]> * path fix Signed-off-by: AlexGrinch <[email protected]> * fixing path Signed-off-by: AlexGrinch <[email protected]> * updated tests to spm Signed-off-by: AlexGrinch <[email protected]> * updated Jenkinsfile Signed-off-by: AlexGrinch <[email protected]> * new model with spm in tests Signed-off-by: AlexGrinch <[email protected]> * yttm removed Signed-off-by: AlexGrinch <[email protected]> * updated aayn config Signed-off-by: AlexGrinch <[email protected]> --------- Signed-off-by: AlexGrinch <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Piotr Żelasko <[email protected]> * Fixed missing copy import in rnnt_decoder.py (#8580) * Added copy import to rnnt_decoding.py Signed-off-by: Isaac McFadyen <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Isaac McFadyen <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Piotr Żelasko <[email protected]> * Fix bug in RNNT Joint WER calculation for fused batch (#8587) Signed-off-by: smajumdar <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Fixed Context Parallel HtoD sync (#8557) * Fixed cp HtoD sync Signed-off-by: Selvaraj Anandaraj <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Piotr Żelasko <[email protected]> * change default and add key to config files (#8594) Signed-off-by: Chen Cui <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Fix triton import guards (#8552) * Fix triton import guards Signed-off-by: Michal Futrega <[email protected]> * Update attention.py Signed-off-by: Michal Futrega <[email protected]> --------- Signed-off-by: Michal Futrega <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Add config key for dropout position in LoRA adapter (#8583) Signed-off-by: Michal Futrega <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * fix ia3 mlp infused adapter (#8597) Signed-off-by: Chen Cui <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Prevent Redundant Gather for LoRA Sequence Parallel (#8602) * enable layernorm output gathered Signed-off-by: Chen Cui <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Piotr Żelasko <[email protected]> * Accelerate `transcribe_speech.py` for short-form data: pre-sorting support (#8564) * POC using bucketing in transcribe_speech.py Signed-off-by: Piotr Żelasko <[email protected]> * extend to multi task aed Signed-off-by: Piotr Żelasko <[email protected]> * fixes for aed multi task text/lang field selectors Signed-off-by: Piotr Żelasko <[email protected]> * remove assert Signed-off-by: Piotr Żelasko <[email protected]> * fix Signed-off-by: Piotr Żelasko <[email protected]> * expose option for bucket buffer size Signed-off-by: Piotr Żelasko <[email protected]> * fixes, ctc support Signed-off-by: Piotr Żelasko <[email protected]> * support pre-sorting manifests in transcribe_speech.py Signed-off-by: Piotr Żelasko <[email protected]> * cleanup Signed-off-by: Piotr Żelasko <[email protected]> * reorder transcriptions back to original manifest order Signed-off-by: Piotr Żelasko <[email protected]> * remove bucketing entirely Signed-off-by: Piotr Żelasko <[email protected]> * code review changes Signed-off-by: Piotr Żelasko <[email protected]> * code review changes--amend Signed-off-by: Piotr Żelasko <[email protected]> * refactor text_field/lang_field passing Signed-off-by: Piotr Żelasko <[email protected]> * Fix reordering bug; disable presorting for multi task for now Signed-off-by: Piotr Żelasko <[email protected]> * Add support for presort + multi task model Signed-off-by: Piotr Żelasko <[email protected]> * Code reviews Signed-off-by: Piotr Żelasko <[email protected]> * Fix jenkins tests, add user-friendly error msg for canary Signed-off-by: Piotr Żelasko <[email protected]> --------- Signed-off-by: Piotr Żelasko <[email protected]> * fix tests Signed-off-by: Piotr Żelasko <[email protected]> * Bump min required lhotse version Signed-off-by: Piotr Żelasko <[email protected]> * Add some documentation about this config format and the multimodal features Signed-off-by: Piotr Żelasko <[email protected]> * Add caution about multiple shards Signed-off-by: Piotr Żelasko <[email protected]> * Address Tom's code review Signed-off-by: Piotr Żelasko <[email protected]> * Add copyright header Signed-off-by: Piotr Żelasko <[email protected]> * Fix (hopefully) issue with forced ascii encoding in CI Signed-off-by: Piotr Żelasko <[email protected]> * Support resolving input_cfg path into config contents Signed-off-by: Piotr Żelasko <[email protected]> * Code review changes in docs Signed-off-by: Piotr Żelasko <[email protected]> * Fix unicode decode error Signed-off-by: Piotr Żelasko <[email protected]> --------- Signed-off-by: Piotr Żelasko <[email protected]> Signed-off-by: w4-jinhyeonkim <[email protected]> Signed-off-by: AlexGrinch <[email protected]> Signed-off-by: Isaac McFadyen <[email protected]> Signed-off-by: smajumdar <[email protected]> Signed-off-by: Selvaraj Anandaraj <[email protected]> Signed-off-by: Chen Cui <[email protected]> Signed-off-by: Michal Futrega <[email protected]> Co-authored-by: w4-jinhyeonkim <[email protected]> Co-authored-by: Aleksey Grinchuk (Oleksii Hrinchuk) <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Isaac McFadyen <[email protected]> Co-authored-by: Somshubra Majumdar <[email protected]> Co-authored-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: Chen Cui <[email protected]> Co-authored-by: Michal Futrega <[email protected]> Co-authored-by: Pablo Garay <[email protected]> Signed-off-by: Ao Tang <[email protected]>
rohitrango
pushed a commit
to rohitrango/NeMo
that referenced
this pull request
Jun 25, 2024
Signed-off-by: smajumdar <[email protected]>
rohitrango
pushed a commit
to rohitrango/NeMo
that referenced
this pull request
Jun 25, 2024
…t-only) dataloading (NVIDIA#8581) * wip Signed-off-by: Piotr Żelasko <[email protected]> * Partially working config groups Signed-off-by: Piotr Żelasko <[email protected]> * Working test with abasic group in the input config Signed-off-by: Piotr Żelasko <[email protected]> * Working test with nested groups in input config Signed-off-by: Piotr Żelasko <[email protected]> * Working test with specifying a YAML path for input_cfg Signed-off-by: Piotr Żelasko <[email protected]> * a very rough example of text dataloading via lhotse Signed-off-by: Piotr Żelasko <[email protected]> * Cleaner integration of multimodal audio/text loading that allows to control the effective audio vs text size (requires latest lhotse) Signed-off-by: Piotr Żelasko <[email protected]> * remove obsolete test Signed-off-by: Piotr Żelasko <[email protected]> * Fix an import in export_utils.py (NVIDIA#8571) Signed-off-by: w4-jinhyeonkim <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Yttm deprecation (NVIDIA#8322) * yttm deprecation init commit Signed-off-by: AlexGrinch <[email protected]> * removed tests Signed-off-by: AlexGrinch <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * bug fix Signed-off-by: AlexGrinch <[email protected]> * path fix Signed-off-by: AlexGrinch <[email protected]> * fixing path Signed-off-by: AlexGrinch <[email protected]> * updated tests to spm Signed-off-by: AlexGrinch <[email protected]> * updated Jenkinsfile Signed-off-by: AlexGrinch <[email protected]> * new model with spm in tests Signed-off-by: AlexGrinch <[email protected]> * yttm removed Signed-off-by: AlexGrinch <[email protected]> * updated aayn config Signed-off-by: AlexGrinch <[email protected]> --------- Signed-off-by: AlexGrinch <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Piotr Żelasko <[email protected]> * Fixed missing copy import in rnnt_decoder.py (NVIDIA#8580) * Added copy import to rnnt_decoding.py Signed-off-by: Isaac McFadyen <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Isaac McFadyen <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Piotr Żelasko <[email protected]> * Fix bug in RNNT Joint WER calculation for fused batch (NVIDIA#8587) Signed-off-by: smajumdar <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Fixed Context Parallel HtoD sync (NVIDIA#8557) * Fixed cp HtoD sync Signed-off-by: Selvaraj Anandaraj <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Piotr Żelasko <[email protected]> * change default and add key to config files (NVIDIA#8594) Signed-off-by: Chen Cui <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Fix triton import guards (NVIDIA#8552) * Fix triton import guards Signed-off-by: Michal Futrega <[email protected]> * Update attention.py Signed-off-by: Michal Futrega <[email protected]> --------- Signed-off-by: Michal Futrega <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Add config key for dropout position in LoRA adapter (NVIDIA#8583) Signed-off-by: Michal Futrega <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * fix ia3 mlp infused adapter (NVIDIA#8597) Signed-off-by: Chen Cui <[email protected]> Signed-off-by: Piotr Żelasko <[email protected]> * Prevent Redundant Gather for LoRA Sequence Parallel (NVIDIA#8602) * enable layernorm output gathered Signed-off-by: Chen Cui <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Piotr Żelasko <[email protected]> * Accelerate `transcribe_speech.py` for short-form data: pre-sorting support (NVIDIA#8564) * POC using bucketing in transcribe_speech.py Signed-off-by: Piotr Żelasko <[email protected]> * extend to multi task aed Signed-off-by: Piotr Żelasko <[email protected]> * fixes for aed multi task text/lang field selectors Signed-off-by: Piotr Żelasko <[email protected]> * remove assert Signed-off-by: Piotr Żelasko <[email protected]> * fix Signed-off-by: Piotr Żelasko <[email protected]> * expose option for bucket buffer size Signed-off-by: Piotr Żelasko <[email protected]> * fixes, ctc support Signed-off-by: Piotr Żelasko <[email protected]> * support pre-sorting manifests in transcribe_speech.py Signed-off-by: Piotr Żelasko <[email protected]> * cleanup Signed-off-by: Piotr Żelasko <[email protected]> * reorder transcriptions back to original manifest order Signed-off-by: Piotr Żelasko <[email protected]> * remove bucketing entirely Signed-off-by: Piotr Żelasko <[email protected]> * code review changes Signed-off-by: Piotr Żelasko <[email protected]> * code review changes--amend Signed-off-by: Piotr Żelasko <[email protected]> * refactor text_field/lang_field passing Signed-off-by: Piotr Żelasko <[email protected]> * Fix reordering bug; disable presorting for multi task for now Signed-off-by: Piotr Żelasko <[email protected]> * Add support for presort + multi task model Signed-off-by: Piotr Żelasko <[email protected]> * Code reviews Signed-off-by: Piotr Żelasko <[email protected]> * Fix jenkins tests, add user-friendly error msg for canary Signed-off-by: Piotr Żelasko <[email protected]> --------- Signed-off-by: Piotr Żelasko <[email protected]> * fix tests Signed-off-by: Piotr Żelasko <[email protected]> * Bump min required lhotse version Signed-off-by: Piotr Żelasko <[email protected]> * Add some documentation about this config format and the multimodal features Signed-off-by: Piotr Żelasko <[email protected]> * Add caution about multiple shards Signed-off-by: Piotr Żelasko <[email protected]> * Address Tom's code review Signed-off-by: Piotr Żelasko <[email protected]> * Add copyright header Signed-off-by: Piotr Żelasko <[email protected]> * Fix (hopefully) issue with forced ascii encoding in CI Signed-off-by: Piotr Żelasko <[email protected]> * Support resolving input_cfg path into config contents Signed-off-by: Piotr Żelasko <[email protected]> * Code review changes in docs Signed-off-by: Piotr Żelasko <[email protected]> * Fix unicode decode error Signed-off-by: Piotr Żelasko <[email protected]> --------- Signed-off-by: Piotr Żelasko <[email protected]> Signed-off-by: w4-jinhyeonkim <[email protected]> Signed-off-by: AlexGrinch <[email protected]> Signed-off-by: Isaac McFadyen <[email protected]> Signed-off-by: smajumdar <[email protected]> Signed-off-by: Selvaraj Anandaraj <[email protected]> Signed-off-by: Chen Cui <[email protected]> Signed-off-by: Michal Futrega <[email protected]> Co-authored-by: w4-jinhyeonkim <[email protected]> Co-authored-by: Aleksey Grinchuk (Oleksii Hrinchuk) <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Isaac McFadyen <[email protected]> Co-authored-by: Somshubra Majumdar <[email protected]> Co-authored-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: Chen Cui <[email protected]> Co-authored-by: Michal Futrega <[email protected]> Co-authored-by: Pablo Garay <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do ?
Fix metric calculation in the case fused batch size > 1 during training / validation calculation.
Previously, the RNNT metric was stateful while the CTC one was not (r1.22.0, r1.23.0)
Therefore this calculation in the RNNT joint for fused operation worked properly. However with the unification of metrics in r1.23.0, a bug was introduced where only the last sub-batch of metrics calculates the scores and does not accumulate.
Collection: [ASR]
Changelog
Before your PR is "Ready for review"
Pre checks:
PR Type: