[runtime] Configurable blank token idx #2366
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 2 commits
February 21, 2024 23:07
zhr1201
commented
Feb 22, 2024
| @@ -153,7 +153,7 @@ std::string ProcessBlank(const std::string& str, bool lowercase) { | |||
| std::wstring_convert<std::codecvt_utf8<wchar_t>, wchar_t> converter; | |||
| std::wstring wsresult = converter.from_bytes(result); | |||
| for (auto& c : wsresult) { | |||
| c = lowercase ? tolower(c, loc) : toupper(c, loc); | |||
| c = lowercase ? tolower(c, loc) : c; | |||
There was a problem hiding this comment.
this changes the --nolowercase behavior, we could also pass another flag if that's what we want to do
zhr1201
changed the title
zhr1201
force-pushed
the
configurable_blank_token_idx
branch
from
d42ce1c to
1757e3b
Compare
zhr1201
force-pushed
the
configurable_blank_token_idx
branch
from
1757e3b to
808ad62
Compare
robin1001
approved these changes
Feb 26, 2024
|
Great job! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Following #2320 which makes feature extraction pipeline compatible with whisper. We realized blank token idx is hard coded #2329, resulting in weird decoding results in the previous PR.
This PR is aiming at fixing that. There might be some places that i missed, can you folks help me do another pass?
Current RTF is 2.xx after warming up, using one core with beam size of 8, on my local macbook. I think the model is probably capable of running at real time with avx512 on multiple cores, even with whisper large. so let's get it to work!
However, there is still some loose ends. Maybe you guys already know the answer and can share some insights.