Skip to content

Refactor JNI to remove casting. - #3103

Merged
csukuangfj merged 6 commits into
k2-fsa:masterfrom
csukuangfj:refactor-jni
Jan 28, 2026
Merged

csukuangfj merged 6 commits into
k2-fsa:masterfrom
csukuangfj:refactor-jni

Conversation

@csukuangfj

@csukuangfj csukuangfj commented Jan 28, 2026 •

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

  • API Improvements

    • Generic array returns replaced with structured result objects across audio, ASR, VAD, keyword-spotting, and denoiser interfaces for safer, clearer data handling.
    • Added a WaveData type for waveform + sample-rate handling.
  • New Features

    • Improved textual formatting of keyword-spotting results for clearer output.
  • Breaking Changes

    • KeywordSpotter constructor now requires an additional configuration parameter.

✏️ Tip: You can customize this high-level summary in your review settings.

@csukuangfj
csukuangfj requested a review from Copilot January 28, 2026 11:10
@coderabbitai

coderabbitai Bot commented Jan 28, 2026 •

Copy link
Copy Markdown

Caution

Review failed

The pull request is closed.

📝 Walkthrough

Walkthrough

The PR replaces untyped JNI Array/Object[] returns with strongly typed Java/Kotlin objects (e.g., WaveData, AudioEvent, various Result classes), updating JNI C++ code, Java wrappers, Kotlin bindings, and example usages to consume the new typed return values.

Changes

Cohort / File(s) Summary
Kotlin example updates
kotlin-api-examples/test_audio_tagging.kt, kotlin-api-examples/test_itn_offline_asr.kt, kotlin-api-examples/test_itn_online_asr.kt, kotlin-api-examples/test_language_id.kt, kotlin-api-examples/test_offline_asr.kt, kotlin-api-examples/test_offline_funasr_nano.kt, kotlin-api-examples/test_offline_medasr_ctc.kt, kotlin-api-examples/test_offline_nemo_canary.kt, kotlin-api-examples/test_offline_omnilingual_asr_ctc.kt, kotlin-api-examples/test_offline_sense_voice_with_hr.kt, kotlin-api-examples/test_offline_speech_denoiser.kt, kotlin-api-examples/test_offline_wenet_ctc.kt, kotlin-api-examples/test_online_asr.kt, kotlin-api-examples/test_speaker_id.kt
Replace objArray unpacking with waveData usage (waveData.samples, waveData.sampleRate); update calls to stream.acceptWaveform and output formatting (audio tagging prints individual event fields).
New Java WaveData and build include
sherpa-onnx/java-api/src/main/java/com/k2fsa/sherpa/onnx/WaveData.java, sherpa-onnx/java-api/Makefile
Add WaveData class (samples, sampleRate, getters, equals, hashCode) and include it in Java build.
Java API wrappers — typed returns
sherpa-onnx/java-api/src/main/java/com/k2fsa/sherpa/onnx/AudioTagging.java, .../KeywordSpotter.java, .../KeywordSpotterResult.java, .../OfflineRecognizer.java, .../OnlineRecognizer.java, .../Vad.java, .../WaveReader.java
Change native declarations to return typed objects (e.g., AudioEvent[], KeywordSpotterResult, OfflineRecognizerResult, OnlineRecognizerResult, SpeechSegment, WaveData) and simplify wrapper code by removing manual Object[] unpacking. Add toString() in KeywordSpotterResult.
JNI implementations — construct typed Java objects
sherpa-onnx/jni/audio-tagging.cc, sherpa-onnx/jni/keyword-spotter.cc, sherpa-onnx/jni/offline-recognizer.cc, sherpa-onnx/jni/online-recognizer.cc, sherpa-onnx/jni/voice-activity-detector.cc, sherpa-onnx/jni/wave-reader.cc
Replace creation/return of intermediate Object[] with direct construction and return of Java objects (AudioEvent, KeywordSpotterResult, OfflineRecognizerResult, OnlineRecognizerResult, SpeechSegment, WaveData). Add null checks and local-ref cleanup.
Kotlin API bindings — typed externals and wrappers
sherpa-onnx/kotlin-api/AudioTagging.kt, sherpa-onnx/kotlin-api/KeywordSpotter.kt, sherpa-onnx/kotlin-api/OfflineRecognizer.kt, sherpa-onnx/kotlin-api/OnlineRecognizer.kt, sherpa-onnx/kotlin-api/Vad.kt, sherpa-onnx/kotlin-api/WaveReader.kt
Update external native signatures from Array<Any> to typed return types (e.g., WaveData, OnlineRecognizerResult, SpeechSegment), simplify wrappers to return native results directly, and expose KeywordSpotter.config as a public property plus toString() for result formatting.

Sequence Diagram(s)

sequenceDiagram
  participant KotlinClient as Kotlin Client
  participant KotlinAPI as Kotlin API (e.g., WaveReader)
  participant JavaAPI as Java wrapper/native bridge
  participant Native as JNI/C++ implementation

  KotlinClient ->> KotlinAPI: readWaveFromFile(filename)
  KotlinAPI ->> JavaAPI: external native readWaveFromFile(filename)
  JavaAPI ->> Native: JNI call into C++
  Native ->> Native: read file, build samples float[] and sampleRate
  Native ->> JavaAPI: construct WaveData(float[], int) jobject and return
  JavaAPI ->> KotlinAPI: return WaveData
  KotlinAPI ->> KotlinClient: return WaveData (samples, sampleRate)
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Poem

🐰 I hopped from arrays to tidy WaveData,
Samples in order, no cast or charade.
JNI knits objects with careful delight,
Kotlin and Java now hold types tight.
Carrots for tests — the code feels so spry!

🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 3.70% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately captures the main objective of the pull request, which is to refactor JNI code to eliminate casting by replacing generic array returns with strongly-typed objects.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@gemini-code-assist

Copy link
Copy Markdown

Summary of Changes

Hello @csukuangfj, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly refactors the Java Native Interface (JNI) layer and its corresponding Java/Kotlin APIs to improve type safety and code clarity. By introducing dedicated data classes for JNI return types and modifying the JNI C++ implementations to directly construct and return these specific Java/Kotlin objects, the need for explicit and potentially unsafe casting of generic Object[] in the Java/Kotlin code has been eliminated. This change streamlines API usage and reduces potential runtime errors.

Highlights

  • New WaveData Class: Introduced a new WaveData class in Java to encapsulate audio samples (float array) and sample rate (integer), providing a structured and type-safe way to handle audio data.
  • WaveReader Refactoring: The WaveReader in both Java and Kotlin APIs has been refactored to directly return instances of the WaveData class from its readWaveFromFile methods, eliminating the need for manual extraction and casting of samples and sample rate from a generic Object[].
  • Type-Safe JNI Returns: The Java Native Interface (JNI) calls for AudioTagging, KeywordSpotter, OfflineRecognizer, OnlineRecognizer, and Vad have been updated to directly return specific, strongly-typed Java/Kotlin objects (e.g., AudioEvent[], KeywordSpotterResult, OfflineRecognizerResult, OnlineRecognizerResult, SpeechSegment) instead of generic Object[]. This removes explicit and unsafe casting in the Java/Kotlin code.
  • JNI C++ Implementation Updates: The underlying C++ JNI implementations (audio-tagging.cc, keyword-spotter.cc, offline-recognizer.cc, online-recognizer.cc, voice-activity-detector.cc, wave-reader.cc) have been modified to construct and return these specific Java/Kotlin objects directly, aligning with the new type-safe API design.
  • Kotlin API Examples Update: All Kotlin API example files have been updated to reflect the new type-safe API, utilizing the WaveData object and direct access to results without casting, improving readability and reducing boilerplate.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@dosubot dosubot Bot added the size:XL This PR changes 500-999 lines, ignoring generated files. label Jan 28, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Fix all issues with AI agents
In `@sherpa-onnx/jni/audio-tagging.cc`:
- Around line 136-168: The FindClass local reference audioEventCls is never
released, causing a JNI local reference leak; after creating the jobjectArray
and before returning from the function that uses audioEventCls (the block that
calls env->FindClass("com/k2fsa/sherpa/onnx/AudioEvent"), env->GetMethodID,
NewObjectArray, and populates it using audioEventCls), call
env->DeleteLocalRef(audioEventCls) once (after the loop and before return) to
free the class local reference; ensure this is done even on error paths where
audioEventCls was obtained (i.e., delete it before any early return if
applicable).

In `@sherpa-onnx/jni/keyword-spotter.cc`:
- Around line 195-215: Add null checks after each JNI lookup in
keyword-spotter.cc: verify env->FindClass("java/lang/String") returned non-null
before using it to create j_tokens, verify
env->FindClass("com/k2fsa/sherpa/onnx/KeywordSpotterResult") returned non-null
and verify env->GetMethodID(result_cls, "<init>",
"(Ljava/lang/String;[Ljava/lang/String;[F)V") returned non-null before calling
the constructor; on any nullptr, clean up any local refs (e.g., delete
j_tokens/j_timestamps/local jstrings if created), optionally throw/clear a Java
exception via env->ThrowNew or return an error/null result, and avoid further
JNI calls that would crash (use the symbols string_cls, j_tokens, j_timestamps,
result_cls, ctor, env->FindClass, env->GetMethodID to locate the checks).
🧹 Nitpick comments (7)
sherpa-onnx/jni/voice-activity-detector.cc (1)

225-231: Consider adding null check after NewObject.

The implementation has good error handling for class/constructor lookup and array allocation, but env->NewObject() can also return nullptr on failure (e.g., if the constructor throws an exception or memory allocation fails). Currently, the function would return nullptr implicitly, which may be acceptable, but adding an explicit check with logging would improve consistency with the other error paths.

♻️ Suggested improvement
   jobject speechSegment =
       env->NewObject(cls, ctor, static_cast<jint>(front.start), samples_arr);
 
+  if (!speechSegment) {
+    SHERPA_ONNX_LOGE("Failed to create SpeechSegment object");
+  }
+
   env->DeleteLocalRef(samples_arr);
   env->DeleteLocalRef(cls);
 
   return speechSegment;
sherpa-onnx/jni/online-recognizer.cc (1)

394-399: Optimize: Move FindClass outside the loop and delete string local refs inside the loop.

FindClass("java/lang/String") is called on every iteration, which is wasteful. Additionally, the jstring objects created by NewStringUTF inside the loop are local references that can accumulate and exhaust the local reference table for large token arrays.

♻️ Suggested refactor
+  jclass stringClass = env->FindClass("java/lang/String");
   jobjectArray tokens = env->NewObjectArray(
-      result.tokens.size(), env->FindClass("java/lang/String"), nullptr);
+      result.tokens.size(), stringClass, nullptr);
   for (size_t i = 0; i < result.tokens.size(); ++i) {
-    env->SetObjectArrayElement(tokens, i,
-                               env->NewStringUTF(result.tokens[i].c_str()));
+    jstring tokenStr = env->NewStringUTF(result.tokens[i].c_str());
+    env->SetObjectArrayElement(tokens, i, tokenStr);
+    env->DeleteLocalRef(tokenStr);
   }
+  env->DeleteLocalRef(stringClass);
sherpa-onnx/kotlin-api/AudioTagging.kt (1)

57-60: Remove unnecessary @Suppress("UNCHECKED_CAST") annotation.

Since the native method now returns Array<AudioEvent> directly and there's no casting in the wrapper, this suppression is no longer needed.

♻️ Suggested fix
-    `@Suppress`("UNCHECKED_CAST")
     fun compute(stream: OfflineStream, topK: Int = -1): Array<AudioEvent> {
         return compute(ptr, stream.ptr, topK)
     }
sherpa-onnx/java-api/src/main/java/com/k2fsa/sherpa/onnx/WaveReader.java (1)

10-13: Consider null-checking the native result.

If readWaveFromFile returns null (e.g., due to file read failure), subsequent calls to getSampleRate() or getSamples() will throw a NullPointerException. While the comment says the program exits on wrong format, there may be other failure modes.

🔧 Suggested defensive check
     public WaveReader(String filename) {
         LibraryLoader.maybeLoad();
-        this.data = readWaveFromFile(filename);
+        this.data = readWaveFromFile(filename);
+        if (this.data == null) {
+            throw new IllegalArgumentException("Failed to read wave file: " + filename);
+        }
     }
sherpa-onnx/jni/offline-recognizer.cc (1)

516-521: Consider adding null checks for class/constructor lookup.

Unlike audio-tagging.cc which checks for null after FindClass and GetMethodID, this code doesn't validate the results. While consistent with the OnlineRecognizer implementation, adding null checks would improve robustness against class loading failures.

🔧 Suggested defensive checks
   // 2. Find the Java class and constructor
   jclass cls = env->FindClass("com/k2fsa/sherpa/onnx/OfflineRecognizerResult");
+  if (cls == nullptr) {
+    SHERPA_ONNX_LOGE("Failed to find class com/k2fsa/sherpa/onnx/OfflineRecognizerResult");
+    return nullptr;
+  }
   jmethodID ctor =
       env->GetMethodID(cls, "<init>",
                        "(Ljava/lang/String;[Ljava/lang/String;[FLjava/lang/"
                        "String;Ljava/lang/String;Ljava/lang/String;[F)V");
+  if (ctor == nullptr) {
+    SHERPA_ONNX_LOGE("Failed to get OfflineRecognizerResult constructor");
+    env->DeleteLocalRef(cls);
+    return nullptr;
+  }
sherpa-onnx/java-api/src/main/java/com/k2fsa/sherpa/onnx/WaveData.java (1)

15-17: Note: getSamples() exposes the internal array.

Returning the internal array directly allows callers to modify the contents, breaking immutability. This is likely intentional for performance in audio processing scenarios, but if immutability is desired, consider returning a defensive copy.

sherpa-onnx/jni/keyword-spotter.cc (1)

221-225: Consider cleaning up additional local references.

string_cls and result_cls are local references that could be released after use. While JNI automatically cleans up local references when the native method returns, explicit cleanup is a good practice for consistency and to avoid hitting the local reference table limit in loops or long-running functions.

♻️ Suggested cleanup
  env->DeleteLocalRef(j_keyword);
  env->DeleteLocalRef(j_tokens);
  env->DeleteLocalRef(j_timestamps);
+ env->DeleteLocalRef(string_cls);
+ env->DeleteLocalRef(result_cls);

  return result_obj;

Comment on lines +136 to 168
// Find the AudioEvent class
jclass audioEventCls = env->FindClass("com/k2fsa/sherpa/onnx/AudioEvent");
if (audioEventCls == nullptr) {
SHERPA_ONNX_LOGE("Failed to find class com/k2fsa/sherpa/onnx/AudioEvent");
return nullptr;
}

// Get the constructor: AudioEvent(String name, int index, float prob)
jmethodID ctor =
env->GetMethodID(audioEventCls, "<init>", "(Ljava/lang/String;IF)V");
if (ctor == nullptr) {
SHERPA_ONNX_LOGE("Failed to get AudioEvent constructor");
return nullptr;
}

// Create a jobjectArray of AudioEvent
jobjectArray obj_arr =
env->NewObjectArray(events.size(), audioEventCls, nullptr);

for (size_t i = 0; i < events.size(); ++i) {
const auto &e = events[i];

jstring name = env->NewStringUTF(e.name.c_str());
jobject event_obj =
env->NewObject(audioEventCls, ctor, name, e.index, e.prob);

env->SetObjectArrayElement(obj_arr, i, event_obj);

env->DeleteLocalRef(name);
env->DeleteLocalRef(event_obj);
}

return obj_arr;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Missing DeleteLocalRef for audioEventCls.

The audioEventCls local reference should be deleted before returning to avoid a local reference leak. This is consistent with the pattern in online-recognizer.cc (from relevant snippets) which deletes cls at the end.

🔧 Suggested fix
     env->DeleteLocalRef(name);
     env->DeleteLocalRef(event_obj);
   }

+  env->DeleteLocalRef(audioEventCls);
+
   return obj_arr;
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// Find the AudioEvent class
jclass audioEventCls = env->FindClass("com/k2fsa/sherpa/onnx/AudioEvent");
if (audioEventCls == nullptr) {
SHERPA_ONNX_LOGE("Failed to find class com/k2fsa/sherpa/onnx/AudioEvent");
return nullptr;
}
// Get the constructor: AudioEvent(String name, int index, float prob)
jmethodID ctor =
env->GetMethodID(audioEventCls, "<init>", "(Ljava/lang/String;IF)V");
if (ctor == nullptr) {
SHERPA_ONNX_LOGE("Failed to get AudioEvent constructor");
return nullptr;
}
// Create a jobjectArray of AudioEvent
jobjectArray obj_arr =
env->NewObjectArray(events.size(), audioEventCls, nullptr);
for (size_t i = 0; i < events.size(); ++i) {
const auto &e = events[i];
jstring name = env->NewStringUTF(e.name.c_str());
jobject event_obj =
env->NewObject(audioEventCls, ctor, name, e.index, e.prob);
env->SetObjectArrayElement(obj_arr, i, event_obj);
env->DeleteLocalRef(name);
env->DeleteLocalRef(event_obj);
}
return obj_arr;
// Find the AudioEvent class
jclass audioEventCls = env->FindClass("com/k2fsa/sherpa/onnx/AudioEvent");
if (audioEventCls == nullptr) {
SHERPA_ONNX_LOGE("Failed to find class com/k2fsa/sherpa/onnx/AudioEvent");
return nullptr;
}
// Get the constructor: AudioEvent(String name, int index, float prob)
jmethodID ctor =
env->GetMethodID(audioEventCls, "<init>", "(Ljava/lang/String;IF)V");
if (ctor == nullptr) {
SHERPA_ONNX_LOGE("Failed to get AudioEvent constructor");
return nullptr;
}
// Create a jobjectArray of AudioEvent
jobjectArray obj_arr =
env->NewObjectArray(events.size(), audioEventCls, nullptr);
for (size_t i = 0; i < events.size(); ++i) {
const auto &e = events[i];
jstring name = env->NewStringUTF(e.name.c_str());
jobject event_obj =
env->NewObject(audioEventCls, ctor, name, e.index, e.prob);
env->SetObjectArrayElement(obj_arr, i, event_obj);
env->DeleteLocalRef(name);
env->DeleteLocalRef(event_obj);
}
env->DeleteLocalRef(audioEventCls);
return obj_arr;
🤖 Prompt for AI Agents
In `@sherpa-onnx/jni/audio-tagging.cc` around lines 136 - 168, The FindClass local
reference audioEventCls is never released, causing a JNI local reference leak;
after creating the jobjectArray and before returning from the function that uses
audioEventCls (the block that calls
env->FindClass("com/k2fsa/sherpa/onnx/AudioEvent"), env->GetMethodID,
NewObjectArray, and populates it using audioEventCls), call
env->DeleteLocalRef(audioEventCls) once (after the loop and before return) to
free the class local reference; ensure this is done even on error paths where
audioEventCls was obtained (i.e., delete it before any early return if
applicable).

Comment thread sherpa-onnx/jni/keyword-spotter.cc

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request is a great refactoring that improves type safety across the JNI boundary for both Java and Kotlin APIs. By returning strongly-typed objects instead of generic Object[] or Array<Any>, the code becomes cleaner, safer, and easier to maintain, eliminating the need for unsafe casting on the client side. The changes are consistently applied across example files, API definitions, and the native JNI implementation. I've found a few potential memory leaks in the JNI layer and a minor inconsistency that should be addressed.

Comment on lines +221 to +223
env->DeleteLocalRef(j_keyword);
env->DeleteLocalRef(j_tokens);
env->DeleteLocalRef(j_timestamps);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

There's a potential local reference leak here. The jclass objects result_cls and string_cls are not released before returning. You should call DeleteLocalRef on them to prevent memory issues.

  env->DeleteLocalRef(j_keyword);
  env->DeleteLocalRef(j_tokens);
  env->DeleteLocalRef(j_timestamps);
  env->DeleteLocalRef(result_cls);
  env->DeleteLocalRef(string_cls);

env->DeleteLocalRef(jemotion);
env->DeleteLocalRef(jevent);
env->DeleteLocalRef(jdurations);
env->DeleteLocalRef(cls);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

There's a potential local reference leak. The jclass object cls is not released before returning. You should call DeleteLocalRef on it to prevent memory issues. I see you are deleting it in the next line, but it is better to group all DeleteLocalRef calls together.

  env->DeleteLocalRef(jevent);
  env->DeleteLocalRef(jdurations);

Comment on lines +396 to 399
for (size_t i = 0; i < result.tokens.size(); ++i) {
env->SetObjectArrayElement(tokens, i,
env->NewStringUTF(result.tokens[i].c_str()));
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

This loop creates a new jstring local reference in each iteration without explicitly deleting it. This can lead to a local reference table overflow and memory leaks. It's safer to explicitly create, use, and delete the reference within the loop.

  for (size_t i = 0; i < result.tokens.size(); ++i) {
    jstring token_str = env->NewStringUTF(result.tokens[i].c_str());
    env->SetObjectArrayElement(tokens, i, token_str);
    env->DeleteLocalRef(token_str);
  }

env->SetObjectArrayElement(obj_arr, 0, samples_arr);
env->SetObjectArrayElement(obj_arr, 1, NewInteger(env, sampling_rate));
// Clean up local refs
env->DeleteLocalRef(samples_arr);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The jclass local reference cls is not being released before the function returns. This can cause a memory leak. You should call env->DeleteLocalRef(cls);.

  env->DeleteLocalRef(samples_arr);
  env->DeleteLocalRef(cls);

Comment on lines +6 to +37
public class WaveData {
private final float[] samples;
private final int sample_rate;

public WaveData(float[] samples, int sample_rate) {
this.samples = samples;
this.sample_rate = sample_rate;
}

public float[] getSamples() {
return samples;
}

public int getSampleRate() {
return sample_rate;
}

@Override
public boolean equals(Object obj) {
if (this == obj) return true;
if (obj == null || getClass() != obj.getClass()) return false;
WaveData other = (WaveData) obj;
return sample_rate == other.sample_rate && Arrays.equals(samples, other.samples);
}

@Override
public int hashCode() {
int result = Arrays.hashCode(samples);
result = 31 * result + sample_rate;
return result;
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The field sample_rate should be renamed to sampleRate to follow Java's camelCase naming convention. This will also make it consistent with the Kotlin WaveData class.

public class WaveData {
    private final float[] samples;
    private final int sampleRate;

    public WaveData(float[] samples, int sampleRate) {
        this.samples = samples;
        this.sampleRate = sampleRate;
    }

    public float[] getSamples() {
        return samples;
    }

    public int getSampleRate() {
        return sampleRate;
    }

    @Override
    public boolean equals(Object obj) {
        if (this == obj) return true;
        if (obj == null || getClass() != obj.getClass()) return false;
        WaveData other = (WaveData) obj;
        return sampleRate == other.sampleRate && Arrays.equals(samples, other.samples);
    }

    @Override
    public int hashCode() {
        int result = Arrays.hashCode(samples);
        result = 31 * result + sampleRate;
        return result;
    }
}

@csukuangfj
csukuangfj requested review from Copilot and removed request for Copilot January 28, 2026 11:25

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@sherpa-onnx/jni/wave-reader.cc`:
- Around line 30-51: The WaveData construction path doesn't guard or clean up
the samples_arr allocation and doesn't check NewObject for null; ensure you
check the result of env->NewFloatArray (samples_arr) for nullptr and return
after logging while calling env->DeleteLocalRef(samples_arr) if allocated, and
also check env->NewObject (obj) for nullptr and on failure delete both
samples_arr and cls before returning; in short, add null checks for
NewFloatArray and NewObject, and call env->DeleteLocalRef for samples_arr and
cls on every early return (and after success) so no local refs leak (refer to
samples_arr, env->NewFloatArray, cls, ctor, env->NewObject, and
env->DeleteLocalRef).
🧹 Nitpick comments (4)
sherpa-onnx/jni/audio-tagging.cc (1)

136-152: Handle error paths to avoid local-ref leaks and null deref.

If GetMethodID fails (Line 145) or NewObjectArray fails (Line 151), cls isn’t released and the code can proceed with a null array. Add cleanup and a null check before the loop.

♻️ Proposed fix
   jclass cls = env->FindClass("com/k2fsa/sherpa/onnx/AudioEvent");
   if (cls == nullptr) {
     SHERPA_ONNX_LOGE("Failed to find class com/k2fsa/sherpa/onnx/AudioEvent");
     return nullptr;
   }

   // Get the constructor: AudioEvent(String name, int index, float prob)
   jmethodID ctor = env->GetMethodID(cls, "<init>", "(Ljava/lang/String;IF)V");
   if (ctor == nullptr) {
     SHERPA_ONNX_LOGE("Failed to get AudioEvent constructor");
+    env->DeleteLocalRef(cls);
     return nullptr;
   }

   // Create a jobjectArray of AudioEvent
   jobjectArray obj_arr = env->NewObjectArray(events.size(), cls, nullptr);
+  if (obj_arr == nullptr) {
+    env->DeleteLocalRef(cls);
+    return nullptr;
+  }
sherpa-onnx/jni/keyword-spotter.cc (2)

192-199: Local reference leak on early return.

j_keyword is created at line 192 before the string_cls null check. If FindClass fails, the function returns nullptr without calling DeleteLocalRef(j_keyword), leaking the reference.

Consider reordering to perform the class lookup first, or adding cleanup before the early return.

♻️ Suggested reordering
- jstring j_keyword = env->NewStringUTF(result.keyword.c_str());
-
  // Convert tokens (std::vector<std::string> -> String[])
  jclass string_cls = env->FindClass("java/lang/String");
  if (string_cls == nullptr) {
    SHERPA_ONNX_LOGE("Failed to find class java/lang/String");
    return nullptr;
  }
+
+ jstring j_keyword = env->NewStringUTF(result.keyword.c_str());

219-231: Resource cleanup missing in error paths.

When result_cls or ctor lookup fails, j_keyword, j_tokens, and j_timestamps are leaked. While these error conditions are rare and references are eventually freed on method return, adding cleanup would make the code more robust.

♻️ Suggested cleanup on error paths
  if (result_cls == nullptr) {
    SHERPA_ONNX_LOGE(
        "Failed to find class com/k2fsa/sherpa/onnx/KeywordSpotterResult");
+   env->DeleteLocalRef(j_keyword);
+   env->DeleteLocalRef(j_tokens);
+   env->DeleteLocalRef(j_timestamps);
+   env->DeleteLocalRef(string_cls);
    return nullptr;
  }

  jmethodID ctor = env->GetMethodID(
      result_cls, "<init>", "(Ljava/lang/String;[Ljava/lang/String;[F)V");

  if (ctor == nullptr) {
    SHERPA_ONNX_LOGE("Failed to get KeywordSpotterResult constructor");
+   env->DeleteLocalRef(j_keyword);
+   env->DeleteLocalRef(j_tokens);
+   env->DeleteLocalRef(j_timestamps);
+   env->DeleteLocalRef(string_cls);
+   env->DeleteLocalRef(result_cls);
    return nullptr;
  }
sherpa-onnx/jni/offline-recognizer.cc (1)

509-515: Add pointer validation guard for getResult.

Line 513 dereferences streamPtr without a guard. Consider wrapping this method in SafeJNI and using ValidatePointer (as done in decode paths) to avoid native crashes on invalid pointers.

💡 Possible refactor
 JNIEXPORT jobject JNICALL
 Java_com_k2fsa_sherpa_onnx_OfflineRecognizer_getResult(JNIEnv *env,
                                                        jobject /*obj*/,
                                                        jlong streamPtr) {
-  auto stream = reinterpret_cast<sherpa_onnx::OfflineStream *>(streamPtr);
-  sherpa_onnx::OfflineRecognitionResult result = stream->GetResult();
+  jobject jresult = nullptr;
+  SafeJNI(env, "OfflineRecognizer_getResult", [&] {
+    if (!ValidatePointer(env, streamPtr, "OfflineRecognizer_getResult",
+                         "OfflineStream pointer is null.")) {
+      return;
+    }
+    auto stream = reinterpret_cast<sherpa_onnx::OfflineStream *>(streamPtr);
+    sherpa_onnx::OfflineRecognitionResult result = stream->GetResult();
     ...
-  return jresult;  // returned object is safe
+  });
+  return jresult;  // returned object is safe
 }

Comment on lines +30 to +51
// Find WaveData class
jclass cls = env->FindClass("com/k2fsa/sherpa/onnx/WaveData");
if (cls == nullptr) {
SHERPA_ONNX_LOGE("Failed to find class com/k2fsa/sherpa/onnx/WaveData");
return nullptr;
}

// Get constructor: WaveData(float[] samples, int sampleRate)
jmethodID ctor = env->GetMethodID(cls, "<init>", "([FI)V");
if (ctor == nullptr) {
SHERPA_ONNX_LOGE("Failed to get WaveData constructor");

env->DeleteLocalRef(cls);
return nullptr;
}

// Create WaveData object
jobject obj = env->NewObject(cls, ctor, samples_arr, sampling_rate);

env->SetObjectArrayElement(obj_arr, 0, samples_arr);
env->SetObjectArrayElement(obj_arr, 1, NewInteger(env, sampling_rate));
// Clean up local refs
env->DeleteLocalRef(samples_arr);
env->DeleteLocalRef(cls);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Harden allocation/cleanup in WaveData construction.

samples_arr is created before the class/ctor lookups; on failures it isn’t released, and there’s no null guard for NewFloatArray. Add a null check and clean up samples_arr on early returns (and after NewObject failure).

🔧 Proposed fix
  jfloatArray samples_arr = env->NewFloatArray(samples.size());
+  if (samples_arr == nullptr) {
+    SHERPA_ONNX_LOGE("Failed to allocate samples array");
+    return nullptr;
+  }
  env->SetFloatArrayRegion(samples_arr, 0, samples.size(), samples.data());

  jclass cls = env->FindClass("com/k2fsa/sherpa/onnx/WaveData");
  if (cls == nullptr) {
    SHERPA_ONNX_LOGE("Failed to find class com/k2fsa/sherpa/onnx/WaveData");
+    env->DeleteLocalRef(samples_arr);
    return nullptr;
  }

  jmethodID ctor = env->GetMethodID(cls, "<init>", "([FI)V");
  if (ctor == nullptr) {
    SHERPA_ONNX_LOGE("Failed to get WaveData constructor");

+    env->DeleteLocalRef(samples_arr);
    env->DeleteLocalRef(cls);
    return nullptr;
  }

  jobject obj = env->NewObject(cls, ctor, samples_arr, sampling_rate);
+  if (obj == nullptr) {
+    env->DeleteLocalRef(samples_arr);
+    env->DeleteLocalRef(cls);
+    return nullptr;
+  }

  env->DeleteLocalRef(samples_arr);
  env->DeleteLocalRef(cls);
🤖 Prompt for AI Agents
In `@sherpa-onnx/jni/wave-reader.cc` around lines 30 - 51, The WaveData
construction path doesn't guard or clean up the samples_arr allocation and
doesn't check NewObject for null; ensure you check the result of
env->NewFloatArray (samples_arr) for nullptr and return after logging while
calling env->DeleteLocalRef(samples_arr) if allocated, and also check
env->NewObject (obj) for nullptr and on failure delete both samples_arr and cls
before returning; in short, add null checks for NewFloatArray and NewObject, and
call env->DeleteLocalRef for samples_arr and cls on every early return (and
after success) so no local refs leak (refer to samples_arr, env->NewFloatArray,
cls, ctor, env->NewObject, and env->DeleteLocalRef).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@csukuangfj
csukuangfj merged commit bd48533 into k2-fsa:master Jan 28, 2026
1 check was pending
@csukuangfj
csukuangfj deleted the refactor-jni branch January 28, 2026 12:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants