Enhanced speaker counting for short audio recordings #2729
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Taejin Park <[email protected]>
Signed-off-by: Taejin Park <[email protected]>
|
This pull request introduces 1 alert when merging a3f70b9 into 132a829 - view on LGTM.com new alerts:
|
nithinraok
requested changes
Aug 26, 2021
| Add randomly generated synthetic embeddings to make eigen analysis more stable. | ||
| We refer to these embeddings as anchor embeddings. | ||
|
|
||
| anchor_sample_n (int): |
There was a problem hiding this comment.
add doc string for emb as well. Is it possible to add some best default values for the remaining arguments
There was a problem hiding this comment.
Added doc string for emb and also added recommended values for the arguments.
| emb_dim = emb.shape[1] | ||
| mean, std_org = np.mean(emb, axis=0), np.std(emb, axis=0) | ||
| new_emb_list = [] | ||
| for _ in range(anchor_spk_n): |
There was a problem hiding this comment.
Can we use torch functions instead of numpy. All these functions can be performed through torch
There was a problem hiding this comment.
Changing np to torch is on hold.
| """ | ||
| est_num_of_spk_list = [] | ||
| for seed in range(random_test_count): | ||
| np.random.seed(seed) |
There was a problem hiding this comment.
same here, move to torch
There was a problem hiding this comment.
Changing np to torch is on hold.
| oracle_num_speakers=None, | ||
| max_num_speaker=8, | ||
| min_samples=6, | ||
| enhanced_count_thres=80, |
There was a problem hiding this comment.
doc string for enhanced count threshold missing
There was a problem hiding this comment.
Added doc string for enhanced_count_thres
| if emb.shape[0] == 1: | ||
| return np.array([0]) | ||
| elif emb.shape[0] < enhanced_count_thres and oracle_num_speakers == None: | ||
| oracle_num_speakers = getEnhancedSpeakerCount(key, emb, cuda) |
There was a problem hiding this comment.
why is this oracle_num_speakers and not est_num_speakers
There was a problem hiding this comment.
also when comparing to None use is None
oracle_num_speakers is None
There was a problem hiding this comment.
Introduced a new variable est_num_of_spk_enhanced to make it clear.
If oracle_num_speakers and est_num_of_speak both doesn't exist, we use est_num_of_spk.
Signed-off-by: Taejin Park <[email protected]>
nithinraok
approved these changes
Aug 30, 2021
Signed-off-by: Taejin Park <[email protected]>
Signed-off-by: Taejin Park <[email protected]>
Signed-off-by: Taejin Park <[email protected]>
Signed-off-by: Taejin Park <[email protected]>
Signed-off-by: Taejin Park <[email protected]>
Signed-off-by: Taejin Park <[email protected]>
Signed-off-by: Taejin Park <[email protected]>
Signed-off-by: Taejin Park <[email protected]>
Signed-off-by: Taejin Park <[email protected]>
Signed-off-by: Taejin Park <[email protected]>
Signed-off-by: Taejin Park <[email protected]>
Signed-off-by: Taejin Park <[email protected]>
Signed-off-by: Taejin Park <[email protected]>
Signed-off-by: Taejin Park <[email protected]>
Signed-off-by: Taejin Park <[email protected]>
…esc_update Signed-off-by: Taejin Park <[email protected]>
paarthneekhara
pushed a commit
to paarthneekhara/NeMo
that referenced
this pull request
Sep 17, 2021
* Update enhanced speaker counting for short samples Signed-off-by: Taejin Park <[email protected]> * Update and doc string change Signed-off-by: Taejin Park <[email protected]> * Reflected PR review comments Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Update enhanced speaker counting for short samples Signed-off-by: Taejin Park <[email protected]> * Update and doc string change Signed-off-by: Taejin Park <[email protected]> * Reflected PR review comments Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> Co-authored-by: Nithin Rao <[email protected]> Signed-off-by: Paarth Neekhara <[email protected]>
jfsantos
pushed a commit
to jfsantos/NeMo
that referenced
this pull request
Nov 19, 2021
* Update enhanced speaker counting for short samples Signed-off-by: Taejin Park <[email protected]> * Update and doc string change Signed-off-by: Taejin Park <[email protected]> * Reflected PR review comments Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Update enhanced speaker counting for short samples Signed-off-by: Taejin Park <[email protected]> * Update and doc string change Signed-off-by: Taejin Park <[email protected]> * Reflected PR review comments Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> * Ran style fix again to fix it Signed-off-by: Taejin Park <[email protected]> Co-authored-by: Nithin Rao <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request is for adding the function getEnhancedSpeakerCount() that performs an enhanced speaker counting for speaker diarization module.
It improves speaker counting accuracy from 50% to 80% especially for short (less than 1 min) recordings.