Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added an alternative way to initialize/load some models (for non-… #1325

Closed
wants to merge 0 commits into from

Conversation

shawl336
Copy link
Contributor

@shawl336 shawl336 commented Sep 7, 2024

Added an alternative way to initialize/load some models (for non-Android codes), tokens, hotwrods and keywords, that initializing/loading from memory buffers. The original usage of model initialization unchanged.
Basically, I just added additional "const char* $_buf_begin, const char* $_buf_end" vars ($ == encoder/decoder/joiner/model), if they are non-nullptr, sherpa will attempt to init the models from the memory starting from "$_buf_start" to "$_buf_end" in priority to the original filename strings.
Also, I added "tokens_buf_str, hotwords_buf_str and keywords_buf_str", if they are non-empty, sherpa will attempt to treat them as the content directly, and using istringstream rather than iftream.
the supported models are:
online models:
OnlineTransducerModel
OnlineParaformerModel
OnlineWenetCtcModel
OnlineZipformer2CtcModel
OnlineNeMoCtcModel

offline models:
  OfflineTransducerModel
  OfflineParaformerModel
  OfflineNemoEncDecCtcModel
  OfflineWhisperModel
  OfflineTdnnModel
  OfflineZipformerCtcModel
  OfflineWenetCtcModel
  OfflineSenseVoiceModel

keywordspotter models:
  KeywordSpotterTransducer

@csukuangfj
Copy link
Collaborator

Could you also add tests for your changes?

const char *joiner;
const char *joiner_buf_begin, *joiner_buf_end; // if non-null, loading the joiner from the buffer in prioriy
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add new fields to the end of existing fields.

Please don't change the order of exsiting fields.

Copy link
Collaborator

@csukuangfj csukuangfj Sep 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest that you use

const char *encoder_buf;
int32_t encoder_buf_len;

@csukuangfj
Copy link
Collaborator

By the way, I suggest that you change one model config per pull-request and add test for the changes.

Otherwise, the pull-request is very large and you need to write lots of tests to cover your changes.

@@ -12,15 +12,25 @@ namespace sherpa_onnx {

struct OnlineTransducerModelConfig {
std::string encoder;
const char *encoder_buf_begin, *encoder_buf_end;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const char *encoder_buf_begin, *encoder_buf_end;
const char *encoder_buf_begin = nullptr;
const char *encoder_buf_end = nullptr;
  1. Please follow our existing code style to define one variable per line.
  2. Please initialize it to nullptr.

Comment on lines 61 to 62
recognizer_config.model_config.transducer.encoder_buf_begin =
SHERPA_ONNX_OR(config->model_config.transducer.encoder_buf_begin, nullptr);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
recognizer_config.model_config.transducer.encoder_buf_begin =
SHERPA_ONNX_OR(config->model_config.transducer.encoder_buf_begin, nullptr);
recognizer_config.model_config.transducer.encoder_buf_begin =
config->model_config.transducer.encoder_buf_begin;

And please change other places.

const char *joiner;
const char *joiner_buf_begin, *joiner_buf_end; // if non-null, loading the joiner from the buffer in prioriy
Copy link
Collaborator

@csukuangfj csukuangfj Sep 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest that you use

const char *encoder_buf;
int32_t encoder_buf_len;

@@ -12,11 +12,15 @@ namespace sherpa_onnx {

struct OfflineNemoEncDecCtcModelConfig {
std::string model;
const char *model_buf_begin, *model_buf_end;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the Validate() method to check that
when model_buf_begin is not nullptr, then model must be empty and vice versa.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants