Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix nmt grpc server, concatdataset for raw text files #2656

Merged
merged 5 commits into from
Aug 14, 2021

Conversation

MaximumEntropy
Copy link
Contributor

This PR fixes

  1. NMT grpc server not setting provided source and target langs correctly
  2. Issue with using a concatenation of multiple raw text files for training NMT models

@MaximumEntropy MaximumEntropy changed the title Fix nmt grpc server and concatdataset for raw text files Fix nmt grpc server, concatdataset for raw text files and NMT CI tests Aug 13, 2021
@lgtm-com
Copy link

lgtm-com bot commented Aug 13, 2021

This pull request fixes 2 alerts when merging 96ccd96 into f3a1215 - view on LGTM.com

fixed alerts:

  • 2 for Modification of parameter with default

@MaximumEntropy MaximumEntropy changed the title Fix nmt grpc server, concatdataset for raw text files and NMT CI tests Fix nmt grpc server, concatdataset for raw text files Aug 13, 2021
@MaximumEntropy MaximumEntropy marked this pull request as ready for review August 13, 2021 21:31
ericharper
ericharper previously approved these changes Aug 13, 2021
Copy link
Collaborator

@ericharper ericharper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks!

@lgtm-com
Copy link

lgtm-com bot commented Aug 13, 2021

This pull request fixes 2 alerts when merging bd6db81 into f3a1215 - view on LGTM.com

fixed alerts:

  • 2 for Modification of parameter with default

Signed-off-by: MaximumEntropy <[email protected]>
@lgtm-com
Copy link

lgtm-com bot commented Aug 13, 2021

This pull request fixes 2 alerts when merging 2988f8f into f3a1215 - view on LGTM.com

fixed alerts:

  • 2 for Modification of parameter with default

@lgtm-com
Copy link

lgtm-com bot commented Aug 13, 2021

This pull request fixes 2 alerts when merging f1c4052 into 12ab790 - view on LGTM.com

fixed alerts:

  • 2 for Modification of parameter with default

Copy link
Collaborator

@ericharper ericharper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks!

@ericharper ericharper merged commit 7780c91 into r1.3.0 Aug 14, 2021
@ericharper ericharper deleted the nmt_bugfixes_and_ci branch August 14, 2021 00:14
ericharper pushed a commit that referenced this pull request Aug 24, 2021
* Fix nmt grpc server and concatdataset for raw text files

Signed-off-by: MaximumEntropy <[email protected]>

* Check if lang direction is provided correctly

Signed-off-by: MaximumEntropy <[email protected]>

* Style fixes

Signed-off-by: MaximumEntropy <[email protected]>
ericharper added a commit that referenced this pull request Aug 24, 2021
* update jenkins branch

Signed-off-by: ericharper <[email protected]>

* update notebooks branch

Signed-off-by: ericharper <[email protected]>

* update package info

Signed-off-by: ericharper <[email protected]>

* update readme

Signed-off-by: ericharper <[email protected]>

* update nemo version for Dockerfile

Signed-off-by: ericharper <[email protected]>

* update notebook branch

Signed-off-by: ericharper <[email protected]>

* Update colab links to Transducer notebooks (#2654)

Signed-off-by: smajumdar <[email protected]>

* Fix nmt grpc server, concatdataset for raw text files (#2656)

* Fix nmt grpc server and concatdataset for raw text files

Signed-off-by: MaximumEntropy <[email protected]>

* Check if lang direction is provided correctly

Signed-off-by: MaximumEntropy <[email protected]>

* Style fixes

Signed-off-by: MaximumEntropy <[email protected]>

* add missing init (#2662)

Signed-off-by: ekmb <[email protected]>

* fix qa inference for single example (#2668)

Signed-off-by: Yang Zhang <[email protected]>

* Fix max symbol per step updating for RNNT (#2672)

* Fix max symbol per step updating for RNNT

Signed-off-by: smajumdar <[email protected]>

* Fix notebooks

Signed-off-by: smajumdar <[email protected]>

* Replaced unfold() with split_view() (#2671)

* Replaced unfold() with split_view()

Signed-off-by: Boris Fomitchev <[email protected]>

* fixed typo

Signed-off-by: Boris Fomitchev <[email protected]>

Co-authored-by: Somshubra Majumdar <[email protected]>

* Correct voice app demo (#2682)

Signed-off-by: smajumdar <[email protected]>

* Import guard (#2692)

* add asr and pynini import guard

Signed-off-by: ekmb <[email protected]>

* remove asrmodel type

Signed-off-by: ekmb <[email protected]>

* remove asrmodel type

Signed-off-by: ekmb <[email protected]>

* fixing branch (#2695)

Signed-off-by: Ghasem Pasandi <[email protected]>

Co-authored-by: Ghasem Pasandi <[email protected]>

* fix for emojis (#2675)

* fix for emojis

Signed-off-by: ekmb <[email protected]>

* remove redundant line

Signed-off-by: ekmb <[email protected]>

* raise error

Signed-off-by: ekmb <[email protected]>

* use app_state

Signed-off-by: ekmb <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* Fix issues with ASR notebooks (#2698)

Signed-off-by: smajumdar <[email protected]>

* Allow non divisible split_size (#2699)

* bugfix

Signed-off-by: Jason <[email protected]>

* bugfix

Signed-off-by: Jason <[email protected]>

* TN fix for corner cases (#2689)

* serial added, weights to common defaults, decimal bug fix

Signed-off-by: ekmb <[email protected]>

* one failing

Signed-off-by: ekmb <[email protected]>

* all tests pass

Signed-off-by: ekmb <[email protected]>

* remove redundant file

Signed-off-by: ekmb <[email protected]>

* fix telephone, add test cases

Signed-off-by: ekmb <[email protected]>

* money fix

Signed-off-by: ekmb <[email protected]>

* clean format

Signed-off-by: ekmb <[email protected]>

* fix edge case of greedy decoding for greedy_batch mode (#2701)

Signed-off-by: smajumdar <[email protected]>

* Remove time macro (#2703)

Signed-off-by: smajumdar <[email protected]>

* Minor FastPitch Fixes (#2697)

* fixes

Signed-off-by: Jason <[email protected]>

* update CI

Signed-off-by: Jason <[email protected]>

* refix

Signed-off-by: Jason <[email protected]>

* Fix ddp error. (#2678)

To avoid "MisconfigurationException: Selected distributed backend ddp is not compatible with an interactive environment." error.

Co-authored-by: ekmb <[email protected]>

* update jenkins

Signed-off-by: ericharper <[email protected]>

* update notebooks

Signed-off-by: ericharper <[email protected]>

* add split_view back

Signed-off-by: ericharper <[email protected]>

Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Boris Fomitchev <[email protected]>
Co-authored-by: Ghasem <[email protected]>
Co-authored-by: Ghasem Pasandi <[email protected]>
Co-authored-by: Jason <[email protected]>
Co-authored-by: khcs <[email protected]>
Co-authored-by: ekmb <[email protected]>
paarthneekhara pushed a commit to paarthneekhara/NeMo that referenced this pull request Sep 17, 2021
* update jenkins branch

Signed-off-by: ericharper <[email protected]>

* update notebooks branch

Signed-off-by: ericharper <[email protected]>

* update package info

Signed-off-by: ericharper <[email protected]>

* update readme

Signed-off-by: ericharper <[email protected]>

* update nemo version for Dockerfile

Signed-off-by: ericharper <[email protected]>

* update notebook branch

Signed-off-by: ericharper <[email protected]>

* Update colab links to Transducer notebooks (NVIDIA#2654)

Signed-off-by: smajumdar <[email protected]>

* Fix nmt grpc server, concatdataset for raw text files (NVIDIA#2656)

* Fix nmt grpc server and concatdataset for raw text files

Signed-off-by: MaximumEntropy <[email protected]>

* Check if lang direction is provided correctly

Signed-off-by: MaximumEntropy <[email protected]>

* Style fixes

Signed-off-by: MaximumEntropy <[email protected]>

* add missing init (NVIDIA#2662)

Signed-off-by: ekmb <[email protected]>

* fix qa inference for single example (NVIDIA#2668)

Signed-off-by: Yang Zhang <[email protected]>

* Fix max symbol per step updating for RNNT (NVIDIA#2672)

* Fix max symbol per step updating for RNNT

Signed-off-by: smajumdar <[email protected]>

* Fix notebooks

Signed-off-by: smajumdar <[email protected]>

* Replaced unfold() with split_view() (NVIDIA#2671)

* Replaced unfold() with split_view()

Signed-off-by: Boris Fomitchev <[email protected]>

* fixed typo

Signed-off-by: Boris Fomitchev <[email protected]>

Co-authored-by: Somshubra Majumdar <[email protected]>

* Correct voice app demo (NVIDIA#2682)

Signed-off-by: smajumdar <[email protected]>

* Import guard (NVIDIA#2692)

* add asr and pynini import guard

Signed-off-by: ekmb <[email protected]>

* remove asrmodel type

Signed-off-by: ekmb <[email protected]>

* remove asrmodel type

Signed-off-by: ekmb <[email protected]>

* fixing branch (NVIDIA#2695)

Signed-off-by: Ghasem Pasandi <[email protected]>

Co-authored-by: Ghasem Pasandi <[email protected]>

* fix for emojis (NVIDIA#2675)

* fix for emojis

Signed-off-by: ekmb <[email protected]>

* remove redundant line

Signed-off-by: ekmb <[email protected]>

* raise error

Signed-off-by: ekmb <[email protected]>

* use app_state

Signed-off-by: ekmb <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* Fix issues with ASR notebooks (NVIDIA#2698)

Signed-off-by: smajumdar <[email protected]>

* Allow non divisible split_size (NVIDIA#2699)

* bugfix

Signed-off-by: Jason <[email protected]>

* bugfix

Signed-off-by: Jason <[email protected]>

* TN fix for corner cases (NVIDIA#2689)

* serial added, weights to common defaults, decimal bug fix

Signed-off-by: ekmb <[email protected]>

* one failing

Signed-off-by: ekmb <[email protected]>

* all tests pass

Signed-off-by: ekmb <[email protected]>

* remove redundant file

Signed-off-by: ekmb <[email protected]>

* fix telephone, add test cases

Signed-off-by: ekmb <[email protected]>

* money fix

Signed-off-by: ekmb <[email protected]>

* clean format

Signed-off-by: ekmb <[email protected]>

* fix edge case of greedy decoding for greedy_batch mode (NVIDIA#2701)

Signed-off-by: smajumdar <[email protected]>

* Remove time macro (NVIDIA#2703)

Signed-off-by: smajumdar <[email protected]>

* Minor FastPitch Fixes (NVIDIA#2697)

* fixes

Signed-off-by: Jason <[email protected]>

* update CI

Signed-off-by: Jason <[email protected]>

* refix

Signed-off-by: Jason <[email protected]>

* Fix ddp error. (NVIDIA#2678)

To avoid "MisconfigurationException: Selected distributed backend ddp is not compatible with an interactive environment." error.

Co-authored-by: ekmb <[email protected]>

* update jenkins

Signed-off-by: ericharper <[email protected]>

* update notebooks

Signed-off-by: ericharper <[email protected]>

* add split_view back

Signed-off-by: ericharper <[email protected]>

Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Boris Fomitchev <[email protected]>
Co-authored-by: Ghasem <[email protected]>
Co-authored-by: Ghasem Pasandi <[email protected]>
Co-authored-by: Jason <[email protected]>
Co-authored-by: khcs <[email protected]>
Co-authored-by: ekmb <[email protected]>
Signed-off-by: Paarth Neekhara <[email protected]>
jfsantos pushed a commit to jfsantos/NeMo that referenced this pull request Nov 19, 2021
* update jenkins branch

Signed-off-by: ericharper <[email protected]>

* update notebooks branch

Signed-off-by: ericharper <[email protected]>

* update package info

Signed-off-by: ericharper <[email protected]>

* update readme

Signed-off-by: ericharper <[email protected]>

* update nemo version for Dockerfile

Signed-off-by: ericharper <[email protected]>

* update notebook branch

Signed-off-by: ericharper <[email protected]>

* Update colab links to Transducer notebooks (NVIDIA#2654)

Signed-off-by: smajumdar <[email protected]>

* Fix nmt grpc server, concatdataset for raw text files (NVIDIA#2656)

* Fix nmt grpc server and concatdataset for raw text files

Signed-off-by: MaximumEntropy <[email protected]>

* Check if lang direction is provided correctly

Signed-off-by: MaximumEntropy <[email protected]>

* Style fixes

Signed-off-by: MaximumEntropy <[email protected]>

* add missing init (NVIDIA#2662)

Signed-off-by: ekmb <[email protected]>

* fix qa inference for single example (NVIDIA#2668)

Signed-off-by: Yang Zhang <[email protected]>

* Fix max symbol per step updating for RNNT (NVIDIA#2672)

* Fix max symbol per step updating for RNNT

Signed-off-by: smajumdar <[email protected]>

* Fix notebooks

Signed-off-by: smajumdar <[email protected]>

* Replaced unfold() with split_view() (NVIDIA#2671)

* Replaced unfold() with split_view()

Signed-off-by: Boris Fomitchev <[email protected]>

* fixed typo

Signed-off-by: Boris Fomitchev <[email protected]>

Co-authored-by: Somshubra Majumdar <[email protected]>

* Correct voice app demo (NVIDIA#2682)

Signed-off-by: smajumdar <[email protected]>

* Import guard (NVIDIA#2692)

* add asr and pynini import guard

Signed-off-by: ekmb <[email protected]>

* remove asrmodel type

Signed-off-by: ekmb <[email protected]>

* remove asrmodel type

Signed-off-by: ekmb <[email protected]>

* fixing branch (NVIDIA#2695)

Signed-off-by: Ghasem Pasandi <[email protected]>

Co-authored-by: Ghasem Pasandi <[email protected]>

* fix for emojis (NVIDIA#2675)

* fix for emojis

Signed-off-by: ekmb <[email protected]>

* remove redundant line

Signed-off-by: ekmb <[email protected]>

* raise error

Signed-off-by: ekmb <[email protected]>

* use app_state

Signed-off-by: ekmb <[email protected]>

Co-authored-by: Eric Harper <[email protected]>

* Fix issues with ASR notebooks (NVIDIA#2698)

Signed-off-by: smajumdar <[email protected]>

* Allow non divisible split_size (NVIDIA#2699)

* bugfix

Signed-off-by: Jason <[email protected]>

* bugfix

Signed-off-by: Jason <[email protected]>

* TN fix for corner cases (NVIDIA#2689)

* serial added, weights to common defaults, decimal bug fix

Signed-off-by: ekmb <[email protected]>

* one failing

Signed-off-by: ekmb <[email protected]>

* all tests pass

Signed-off-by: ekmb <[email protected]>

* remove redundant file

Signed-off-by: ekmb <[email protected]>

* fix telephone, add test cases

Signed-off-by: ekmb <[email protected]>

* money fix

Signed-off-by: ekmb <[email protected]>

* clean format

Signed-off-by: ekmb <[email protected]>

* fix edge case of greedy decoding for greedy_batch mode (NVIDIA#2701)

Signed-off-by: smajumdar <[email protected]>

* Remove time macro (NVIDIA#2703)

Signed-off-by: smajumdar <[email protected]>

* Minor FastPitch Fixes (NVIDIA#2697)

* fixes

Signed-off-by: Jason <[email protected]>

* update CI

Signed-off-by: Jason <[email protected]>

* refix

Signed-off-by: Jason <[email protected]>

* Fix ddp error. (NVIDIA#2678)

To avoid "MisconfigurationException: Selected distributed backend ddp is not compatible with an interactive environment." error.

Co-authored-by: ekmb <[email protected]>

* update jenkins

Signed-off-by: ericharper <[email protected]>

* update notebooks

Signed-off-by: ericharper <[email protected]>

* add split_view back

Signed-off-by: ericharper <[email protected]>

Co-authored-by: Somshubra Majumdar <[email protected]>
Co-authored-by: Sandeep Subramanian <[email protected]>
Co-authored-by: Evelina <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Boris Fomitchev <[email protected]>
Co-authored-by: Ghasem <[email protected]>
Co-authored-by: Ghasem Pasandi <[email protected]>
Co-authored-by: Jason <[email protected]>
Co-authored-by: khcs <[email protected]>
Co-authored-by: ekmb <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants