Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NFA updates #6695

Merged
merged 60 commits into from
Jun 9, 2023
Merged

NFA updates #6695

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
037194b
update V_NEGATIVE_NUM constant to make better use of torch.float32 range
erastorgueva-nv Mar 11, 2023
cb69ccc
adjust backpointers dtype if U_max too large
erastorgueva-nv Mar 11, 2023
0dd8729
Remove print statements
erastorgueva-nv Mar 11, 2023
cceab7c
Remove need for user to specify model_downsample_factor
erastorgueva-nv Mar 11, 2023
f9489d8
change model.cfg.sample_rate to model.cfg.preprocessor.sample_rate
erastorgueva-nv Mar 13, 2023
7b37972
add check to make sure that window_stride is in model.cfg.preprocessor
erastorgueva-nv Mar 13, 2023
0cef35e
reduce memory consumption of backpointers by making them relative ins…
erastorgueva-nv Mar 15, 2023
11f3430
update librosa.get_duration() 'filename' param to 'path'
erastorgueva-nv Mar 15, 2023
9d9b7b2
Do not throw error if 'text' or 'pred_text' are empty and make sure C…
erastorgueva-nv Mar 16, 2023
d916db4
preprocess input text by removing any duplicate spaces and converting…
erastorgueva-nv Mar 16, 2023
643a8ee
Use Utterance dataclass instead of dictionaries for keeping track of …
erastorgueva-nv Apr 4, 2023
2be92bf
Merge branch 'main' into nfa_updates
erastorgueva-nv Apr 4, 2023
0897f33
refactor so can save alignments as ctm and ass format files
erastorgueva-nv Apr 5, 2023
c74a63f
fix bugs for saving character based ASS files and for using pred_text…
erastorgueva-nv Apr 5, 2023
f7c920e
Make token level .ass file use tokens with recovered capitalization
erastorgueva-nv Apr 6, 2023
45e3fb1
Do not try to generate alignment files if text or pred text is empty,…
erastorgueva-nv Apr 6, 2023
a57409d
rename output manifest file to say '_with_output_file_paths.json'
erastorgueva-nv Apr 6, 2023
dbd5232
add flag to resegment ass subtitle file to fill available text space
erastorgueva-nv Apr 7, 2023
10f85e3
Fix bug in resegmentation code
erastorgueva-nv Apr 8, 2023
f1561d4
Fix bug which skipped some utterances if batch_size more than 1
erastorgueva-nv Apr 20, 2023
5ebe1e4
reduce memory requirements by doing torch.gather on a slice of the lo…
erastorgueva-nv Apr 21, 2023
ccda03e
reduce memory requirements by not saving whole v_matrix
erastorgueva-nv Apr 22, 2023
aad4d04
remove any extra spaces in pred_text
erastorgueva-nv May 16, 2023
49031d6
Merge branch 'main' into nfa_updates
erastorgueva-nv May 22, 2023
033a9fd
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 22, 2023
b49ddb7
remove unused list pred_text_all_lines
erastorgueva-nv May 22, 2023
afab69e
support using hybrid Transducer-CTC models for alignment
erastorgueva-nv May 23, 2023
041b18d
Merge branch 'main' into nfa_updates
erastorgueva-nv Jun 6, 2023
623369a
fix typo - add brackets to torch.cuda.is_available()
erastorgueva-nv Jun 6, 2023
2debbc0
make sure token case restoration will work if superscript or subscrip…
erastorgueva-nv Jun 6, 2023
2879cad
Merge branch 'main' into nfa_updates
erastorgueva-nv Jun 6, 2023
918abdc
remove any BOM from input text
erastorgueva-nv Jun 6, 2023
9c91d43
pick out 1st hypotheses if there is a tuple of them
erastorgueva-nv Jun 6, 2023
846957c
Remove print statement
erastorgueva-nv Jun 6, 2023
0940c70
add detail to error message if fail to recover capitalization of tokens
erastorgueva-nv Jun 6, 2023
ed6a4b8
add flag use_local_attention
erastorgueva-nv Jun 6, 2023
412633e
rename additional_ctm_grouping_separator -> additional_segment_groupi…
erastorgueva-nv Jun 6, 2023
ad7df58
update description of additional_segment_grouping_separator
erastorgueva-nv Jun 6, 2023
bd5d274
add simple docstring to get_utt_obj function
erastorgueva-nv Jun 6, 2023
a5f793b
Make docstring for add_t_start_end_to_utt_obj
erastorgueva-nv Jun 6, 2023
47c72d1
update docstrings for add_t_start_end_to_utt_obj and get_batch_variables
erastorgueva-nv Jun 6, 2023
e239375
update README and comments in align.py
erastorgueva-nv Jun 6, 2023
af35b5e
change 'ground truth' -> 'reference text' in documentation
erastorgueva-nv Jun 6, 2023
ecb3ce2
add header
erastorgueva-nv Jun 7, 2023
93ac8f6
add comments to get_utt_obj function
erastorgueva-nv Jun 7, 2023
ce243e0
move constants so they are after imports
erastorgueva-nv Jun 7, 2023
9827ce4
add file description for make_ass_files
erastorgueva-nv Jun 7, 2023
5935ada
get rid of Utterance object's S attribute, and correct tests so they …
erastorgueva-nv Jun 7, 2023
23380c5
Merge branch 'main' into nfa_updates
erastorgueva-nv Jun 7, 2023
8ac4a2f
remove some unused variables
erastorgueva-nv Jun 7, 2023
195f306
remove unused variable model from functions saving output files
erastorgueva-nv Jun 7, 2023
b92c25f
remove unused var minimum_timestamp_duration from make_ass_files func…
erastorgueva-nv Jun 7, 2023
d3a49e5
move minimum_timestamp_duration param to CTMFileConfig
erastorgueva-nv Jun 7, 2023
4f7714c
Merge branch 'main' into nfa_updates
erastorgueva-nv Jun 8, 2023
dab537d
remove unused enumerate and unused import
erastorgueva-nv Jun 8, 2023
7312497
switch reading duration from librosa to soundfile to avoid filename/p…
erastorgueva-nv Jun 8, 2023
76cd1b3
Merge branch 'main' into nfa_updates
erastorgueva-nv Jun 8, 2023
6b7959b
Merge branch 'main' into nfa_updates
erastorgueva-nv Jun 8, 2023
0b6c5f7
Merge branch 'main' into nfa_updates
erastorgueva-nv Jun 8, 2023
93c0d69
Merge branch 'main' into nfa_updates
erastorgueva-nv Jun 8, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 44 additions & 25 deletions tools/nemo_forced_aligner/README.md
View file
Open in desktop

Large diffs are not rendered by default.

186 changes: 89 additions & 97 deletions tools/nemo_forced_aligner/align.py
View file
Open in desktop

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion tools/nemo_forced_aligner/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
nemo_toolkit[all]
pytest
prettyprinter # for testing
pytest # for testing
121 changes: 0 additions & 121 deletions tools/nemo_forced_aligner/tests/test_add_t_start_end_to_boundary_info.py
View file
Open in desktop

This file was deleted.

Loading