Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add bf16 inference support and fix seq_len stft issue #7338

Merged
merged 2 commits into from
Aug 28, 2023
Merged

add bf16 inference support and fix seq_len stft issue #7338

merged 2 commits into from
Aug 28, 2023

Conversation

nithinraok
Copy link
Collaborator

What does this PR do ?

add bf16 inference support and fix seq_len stft issue

Collection: ASR

Changelog

  • torch.floor_divide will make sure to not go into round off issues at higher precisions
  • add support for amp_dtype during transcribing

Usage

python speech_to_text_eval.py dataset_manifest=<name> amp=True amp_dtype=bfloat16

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?
  • Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
    • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

  • New Feature
  • Bugfix
  • Documentation

add bf16 inference support and fix seq_len stft issue
e0f62c9 
Signed-off-by: Nithin Rao Koluguri <nithinraok>
@github-actions github-actions bot added the ASR label Aug 28, 2023
titu1994
titu1994 previously approved these changes Aug 28, 2023
View reviewed changes
Copy link
Collaborator

@titu1994 titu1994 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch for the off by one bug !

typo
772e231 
Signed-off-by: Nithin Rao Koluguri <nithinraok>
@titu1994 titu1994 merged commit 78c1c4a into main Aug 28, 2023
12 of 14 checks passed
@titu1994 titu1994 deleted the fix_stft_add_bf16 branch August 28, 2023 19:46
rohitrango pushed a commit to rohitrango/NeMo that referenced this pull request Jun 25, 2024
@nithinraok
add bf16 inference support and fix seq_len stft issue (NVIDIA#7338)
f21ef14 
* add bf16 inference support and fix seq_len stft issue

Signed-off-by: Nithin Rao Koluguri <nithinraok>

* typo

Signed-off-by: Nithin Rao Koluguri <nithinraok>

---------

Signed-off-by: Nithin Rao Koluguri <nithinraok>
Co-authored-by: Nithin Rao Koluguri <nithinraok>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@titu1994 titu1994 titu1994 approved these changes

Assignees
No one assigned
Labels
ASR
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@nithinraok @titu1994