Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to split AMI segments similar to Kaldi #889

Merged
merged 1 commit into from
Nov 12, 2022

Conversation

desh2608
Copy link
Collaborator

Kaldi AMI data preparation includes splitting long segments based on full-stops and commas. This is useful for training ASR models since otherwise we would throw away a lot of data. Here are some statistics before and after adding this option:

# segments Train Dev Test
Kaldi 105149 13059 12612
Lhotse (w/o splitting) 65557 8665 7490
Lhotse (w/ splitting) 103377 12614 12121

The remaining difference may be because we don't "over-segment" the segments if they are already under the max words.

@desh2608
Copy link
Collaborator Author

(@HuangZiliAndy and I have both observed that decoding WERs can be significantly better with the segment splitting.)

@pzelasko
Copy link
Collaborator

Great!

@pzelasko pzelasko merged commit defdafc into lhotse-speech:master Nov 12, 2022
@pzelasko pzelasko added this to the v1.10 milestone Nov 12, 2022
@desh2608 desh2608 deleted the recipe/ami_split_segs branch November 2, 2023 19:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants