Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Hybrid conformer export #6995

Merged
merged 1 commit into from
Jul 7, 2023
Merged

Hybrid conformer export #6995

merged 1 commit into from
Jul 7, 2023

Conversation

github-actions[bot]
Copy link
Contributor

@github-actions github-actions bot commented Jul 7, 2023

What does this PR do ?

Allows export of Hybrid Models with either RNNT decoder (in two parts) or with CTC decoder as single encoder-decoder module.
Usage: python scripts/export.py file.nemo file.onnx --config decoder_type=[ctc|rnnt]

Collection: [ASR]

Changelog

  • Add specific line by line info of high level changes in this PR.

Usage

python scripts/export.py file.nemo file.onnx --config decoder_type=[ctc|rnnt]

Add a code snippet demonstrating how to use this


# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)

@borisfom @web-flow
Hybrid conformer export (#6983)
d24780e 
* Implemented generic kv-pair setting of export_config from args

Signed-off-by: Boris Fomitchev <[email protected]>

* Hybrid conformer export

Signed-off-by: Boris Fomitchev <[email protected]>

* Hybrid decoder export

Signed-off-by: Boris Fomitchev <[email protected]>

* Cleanup

Signed-off-by: Boris Fomitchev <[email protected]>

* Changed from **kwargs

Signed-off-by: Boris Fomitchev <[email protected]>

* Docstring

Signed-off-by: Boris Fomitchev <[email protected]>

* Docs added

Signed-off-by: Boris Fomitchev <[email protected]>

* Stringify args

Signed-off-by: Boris Fomitchev <[email protected]>

* Added docs for ASR export configs

Signed-off-by: Boris Fomitchev <[email protected]>

* lowercase ctc

Signed-off-by: Boris Fomitchev <[email protected]>

---------

Signed-off-by: Boris Fomitchev <[email protected]>
@github-actions github-actions bot added ASR cherry-pick core Changes to NeMo Core labels Jul 7, 2023
@github-actions github-actions bot requested a review from borisfom July 7, 2023 21:32
@titu1994 titu1994 merged commit 94e1efa into main Jul 7, 2023
@titu1994 titu1994 deleted the cherry-pick-main-b5cb7831ddac10c6a6be57d21cfd09dbc3b10c36 branch July 7, 2023 21:52
gshennvm pushed a commit that referenced this pull request Jul 12, 2023
@github-actions @borisfom @gshennvm
Hybrid conformer export (#6983) (#6995)
d904b60 
* Implemented generic kv-pair setting of export_config from args



* Hybrid conformer export



* Hybrid decoder export



* Cleanup



* Changed from **kwargs



* Docstring



* Docs added



* Stringify args



* Added docs for ASR export configs



* lowercase ctc



---------

Signed-off-by: Boris Fomitchev <[email protected]>
Co-authored-by: Boris Fomitchev <[email protected]>
Signed-off-by: Gerald Shen <[email protected]>
gshennvm pushed a commit that referenced this pull request Jul 12, 2023
@github-actions @borisfom @gshennvm
Hybrid conformer export (#6983) (#6995)
b04bb98 
* Implemented generic kv-pair setting of export_config from args



* Hybrid conformer export



* Hybrid decoder export



* Cleanup



* Changed from **kwargs



* Docstring



* Docs added



* Stringify args



* Added docs for ASR export configs



* lowercase ctc



---------

Signed-off-by: Boris Fomitchev <[email protected]>
Co-authored-by: Boris Fomitchev <[email protected]>
Signed-off-by: Gerald Shen <[email protected]>
ericharper added a commit that referenced this pull request Jul 13, 2023
@markelsanz14 @pre-commit-ci
@kit1980 @gshennvm @github-actions @yidong72 @Zhilin123 @vadimkantorov @borisfom @titu1994 @Kipok @rlangman @trias702 @ericharper @arendu @MaximumEntropy @ekmb
Add end_strings to SamplingParams (#6986)
f7e33fc 
* Add end_strings to SamplingParams

Signed-off-by: Gerald Shen <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: Gerald Shen <[email protected]>

* Add end_strings to megatron_gpt_inference.yaml

Signed-off-by: Gerald Shen <[email protected]>

* Add end_strings to sampling params

Signed-off-by: Gerald Shen <[email protected]>

* Remove extra_id_1 from default end_strings

Signed-off-by: Gerald Shen <[email protected]>

* Fix require_grad typos (#6930)

Signed-off-by: Sergii Dymchenko <[email protected]>
Signed-off-by: Gerald Shen <[email protected]>

* fix syntax error

Signed-off-by: Gerald Shen <[email protected]>

* fix the mpt chatbot (#6957) (#6968)

Signed-off-by: Yi Dong <[email protected]>
Co-authored-by: Yi Dong <[email protected]>
Signed-off-by: Gerald Shen <[email protected]>

* add support for max_total_length=4096 for 43b (#6763)

* add support for max_total_length=4096 for 43b

Signed-off-by: Zhilin Wang <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Zhilin Wang <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Gerald Shen <[email protected]>

* rnnt_greedy_decoding.py: typos? auto-repressively -> auto-regressively (#6989)

Signed-off-by: Vadim Kantorov <[email protected]>
Signed-off-by: Gerald Shen <[email protected]>

* Cache handling without input tensors mutation (#6980) (#6996)

* Cache handling without input tensors mutation



* Cleanup



* Cleanup#2



* Cleanup#3



---------

Signed-off-by: Boris Fomitchev <[email protected]>
Co-authored-by: Boris Fomitchev <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Signed-off-by: Gerald Shen <[email protected]>

* Hybrid conformer export (#6983) (#6995)

* Implemented generic kv-pair setting of export_config from args



* Hybrid conformer export



* Hybrid decoder export



* Cleanup



* Changed from **kwargs



* Docstring



* Docs added



* Stringify args



* Added docs for ASR export configs



* lowercase ctc



---------

Signed-off-by: Boris Fomitchev <[email protected]>
Co-authored-by: Boris Fomitchev <[email protected]>
Signed-off-by: Gerald Shen <[email protected]>

* Fixing an issue with confidence ensembles (#6987) (#7004)

* Bug fix for the confidence ensembles



* Relax constraints for the test



---------

Signed-off-by: Igor Gitman <[email protected]>
Co-authored-by: Igor Gitman <[email protected]>
Signed-off-by: Gerald Shen <[email protected]>

* [TTS] Add cosine distance option to TTS aligner (#6806)

* [TTS] Add cosine distance option to TTS aligner

Signed-off-by: Ryan <[email protected]>

* [TTS] Update aligner comments

Signed-off-by: Ryan <[email protected]>

---------

Signed-off-by: Ryan <[email protected]>
Signed-off-by: Gerald Shen <[email protected]>

* Minor MPT-7B fixes and creation script update (#6982)

* Initial commit of minor MPT-7B fixes

Signed-off-by: Daniel Egert <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Daniel Egert <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Gerald Shen <[email protected]>

* Change Jenkins timeout (#6997)

* change timeout

Signed-off-by: ericharper <[email protected]>

* change to 8 hours

Signed-off-by: ericharper <[email protected]>

---------

Signed-off-by: ericharper <[email protected]>
Signed-off-by: Gerald Shen <[email protected]>

* remove hard coded input and output fields (#7008)

* remove hard coded input and output fields

Signed-off-by: arendu <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: arendu <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Gerald Shen <[email protected]>

* RoPE length extrapolation with interpolation (#7005)

* Push changes

Signed-off-by: MaximumEntropy <[email protected]>

* Fixes

Signed-off-by: MaximumEntropy <[email protected]>

* add continue training script

Signed-off-by: MaximumEntropy <[email protected]>

* [WIP] nonlinear interp

Signed-off-by: MaximumEntropy <[email protected]>

* Fix

Signed-off-by: MaximumEntropy <[email protected]>

* override encoder_seq_len

Signed-off-by: MaximumEntropy <[email protected]>

* Remove nonlinear

Signed-off-by: MaximumEntropy <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* sft with pi (#7006)

* sft with pi

Signed-off-by: Evelina <[email protected]>

* update values only if not None"

Signed-off-by: Evelina <[email protected]>

---------

Signed-off-by: Evelina <[email protected]>

* Address comments

Signed-off-by: MaximumEntropy <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add info

Signed-off-by: MaximumEntropy <[email protected]>

* Empty

Signed-off-by: MaximumEntropy <[email protected]>

---------

Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: Evelina <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Evelina <[email protected]>
Signed-off-by: Gerald Shen <[email protected]>

* use proper config

Signed-off-by: Gerald Shen <[email protected]>

* Add end_strings to SamplingParams

Signed-off-by: Gerald Shen <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: Gerald Shen <[email protected]>

* Add end_strings to megatron_gpt_inference.yaml

Signed-off-by: Gerald Shen <[email protected]>

* Add end_strings to sampling params

Signed-off-by: Gerald Shen <[email protected]>

* Remove extra_id_1 from default end_strings

Signed-off-by: Gerald Shen <[email protected]>

* fix syntax error

Signed-off-by: Gerald Shen <[email protected]>

* use proper config

Signed-off-by: Gerald Shen <[email protected]>

---------

Signed-off-by: Gerald Shen <[email protected]>
Signed-off-by: Sergii Dymchenko <[email protected]>
Signed-off-by: Yi Dong <[email protected]>
Signed-off-by: Zhilin Wang <[email protected]>
Signed-off-by: Vadim Kantorov <[email protected]>
Signed-off-by: Boris Fomitchev <[email protected]>
Signed-off-by: Igor Gitman <[email protected]>
Signed-off-by: Ryan <[email protected]>
Signed-off-by: Daniel Egert <[email protected]>
Signed-off-by: ericharper <[email protected]>
Signed-off-by: arendu <[email protected]>
Signed-off-by: MaximumEntropy <[email protected]>
Signed-off-by: Evelina <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Sergii Dymchenko <[email protected]>
Co-authored-by: Gerald Shen <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Yi Dong <[email protected]>
Co-authored-by: Zhilin Wang <[email protected]>
Co-authored-by: Vadim Kantorov <[email protected]>
Co-authored-by: Boris Fomitchev <[email protected]>
Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Igor Gitman <[email protected]>
Co-authored-by: Ryan Langman <[email protected]>
Co-authored-by: trias702 <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Adi Renduchintala <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Evelina <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@borisfom borisfom Awaiting requested review from borisfom

Assignees
No one assigned
Labels
ASR cherry-pick core Changes to NeMo Core
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@titu1994 @borisfom