Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add context biasing for mobile #568

Merged
merged 6 commits into from
Feb 1, 2024

Conversation

ductranminh
Copy link
Contributor

I implement context biasing (hot words) for android/ios frontend by calling:

Reset(hotWords)

Copy link
Collaborator

@csukuangfj csukuangfj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Left some minor comments.

@@ -85,14 +85,17 @@ class SherpaOnnx(
acceptWaveform(ptr, samples, sampleRate)

fun inputFinished() = inputFinished(ptr)
fun reset(recreate: Boolean = false) = reset(ptr, recreate = recreate)
fun reset(recreate: Boolean = false, hotWords: String = "") = reset(ptr, recreate, hotWords)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you also update

fun reset(recreate: Boolean = false) = reset(ptr, recreate = recreate)

@@ -85,14 +85,17 @@ class SherpaOnnx(
acceptWaveform(ptr, samples, sampleRate)

fun inputFinished() = inputFinished(ptr)
fun reset(recreate: Boolean = false) = reset(ptr, recreate = recreate)
fun reset(recreate: Boolean = false, hotWords: String = "") = reset(ptr, recreate, hotWords)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
fun reset(recreate: Boolean = false, hotWords: String = "") = reset(ptr, recreate, hotWords)
fun reset(recreate: Boolean = false, hotwords: String = "") = reset(ptr, recreate, hotwords)

auto stream = recognizer_.CreateStream(keywords);
// Set new keywords failed, the stream_ will not be updated.
if (stream != nullptr) {
stream_ = std::move(stream);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you also use SHERPA_ONNX_LOGE() to print some logs here saying it failed to set the hotwords?

Comment on lines 1526 to 1527
std::string keywords_str = p_keywords;
model->Reset(recreate, keywords_str);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
std::string keywords_str = p_keywords;
model->Reset(recreate, keywords_str);
model->Reset(recreate, p_keywords);

You can pass p_keywords directly to the function.

Copy link
Contributor Author

@ductranminh ductranminh Feb 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But p_keywords is const char * while the reset method accept std::string ? It's incorrect type

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But p_keywords is const char * while the reset method accept std::string ? It's incorrect type

Does it cause compiler errors? If yes, please post the error logs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you're right, it works normally

}

words.withCString { cString in
stream = CreateOnlineStreamWithHotwords(recognizer, cString)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please free the previous stream to avoid memory leak.

@ductranminh
Copy link
Contributor Author

@csukuangfj I updated the fixes

@csukuangfj
Copy link
Collaborator

@csukuangfj I updated the fixes

Thanks! Only one comment left now. (I just added one.)

/// If hotWords is an empty string, it just recreates the decoding stream
/// If hotWords is not empty, it will create a new decoding stream with
/// the given hotWords appended to the default hotWords.
func reset(hotWords: String? = nil) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
func reset(hotWords: String? = nil) {
func reset(hotwords: String? = nil) {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment on lines 84 to 88
if (recreate) {
stream_ = recognizer_.CreateStream();
} else {
} else {
recognizer_.Reset(stream_.get());
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you indent the statements?

You can run

cd /path/to/sherpa-onnx
./scripts/check_style_cpplint.sh 
./scripts/check_style_cpplint.sh 1
./scripts/check_style_cpplint.sh 2

to check your code style.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you indent the statements?

You can run

cd /path/to/sherpa-onnx
./scripts/check_style_cpplint.sh 
./scripts/check_style_cpplint.sh 1
./scripts/check_style_cpplint.sh 2

to check your code style.

Done!
All code style checks passed

Copy link
Collaborator

@csukuangfj csukuangfj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Looks good to me except for the C++ code style.

@csukuangfj
Copy link
Collaborator

Thank you again for your contribution!

@csukuangfj csukuangfj merged commit 665b869 into k2-fsa:master Feb 1, 2024
142 of 146 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants