Repository navigation
Add Swift API for Google MedASR model - #2947
Conversation
dosubot
Bot
added
the
size:M
📝 WalkthroughWalkthroughThis PR adds support for a new MED ASR (Multi-Encoder Decoder ASR) CTC model type to Sherpa ONNX's Swift API examples. Changes include configuration helpers, a Swift implementation example, an automated build and execution script, expanded test infrastructure, and gitignore updates. Changes
Sequence DiagramsequenceDiagram
actor User
participant BuildScript as run-medasr-ctc-<br/>asr.sh
participant SwiftCompiler as Swift Compiler
participant SherpaLib as Sherpa ONNX<br/>Library
participant App as medasr-ctc<br/>Application
participant AudioFile as Audio File
User->>BuildScript: Execute script
BuildScript->>BuildScript: Check Swift build<br/>directory exists
BuildScript->>BuildScript: Verify tokens.txt<br/>(download if needed)
alt Executable missing
BuildScript->>SwiftCompiler: Invoke swiftc with<br/>Sherpa ONNX linkage
SwiftCompiler->>SherpaLib: Link against<br/>libsherpa-onnx
SwiftCompiler-->>BuildScript: Produce binary
BuildScript->>BuildScript: Strip binary
end
BuildScript->>BuildScript: Set DYLD_LIBRARY_PATH
BuildScript->>App: Execute medasr-ctc
App->>App: Configure MED ASR model<br/>& feature settings
App->>SherpaLib: Create offline recognizer
SherpaLib-->>App: Recognizer ready
App->>AudioFile: Load WAV samples
AudioFile-->>App: Audio data
App->>SherpaLib: Decode audio stream
SherpaLib-->>App: Transcription + timestamps
App->>User: Print results
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: defaults Review profile: CHILL Plan: Pro 📒 Files selected for processing (5)
🧰 Additional context used🧬 Code graph analysis (1)swift-api-examples/medasr-ctc.swift (1)
🔇 Additional comments (6)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary of ChangesHello @csukuangfj, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the Swift API by introducing support for the Google MedASR model. It provides the necessary Swift bindings and a comprehensive example, allowing developers to seamlessly integrate and leverage the MedASR model for offline automatic speech recognition within their Swift projects. This expansion broadens the capabilities of the Sherpa Onnx Swift API, offering more options for ASR tasks. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces Swift API support for the Google MedASR model. It includes a new example (medasr-ctc.swift), a script to run it (run-medasr-ctc-asr.sh), updates to the Swift wrapper (SherpaOnnx.swift), and integration into the CI test script. The changes are well-structured and follow the existing patterns in the repository. My review focuses on improving the maintainability and code style of the newly added example and its corresponding run script by refactoring repeated strings into variables and using more idiomatic Swift constructs.
| func run() { | ||
| let model = | ||
| "./sherpa-onnx-medasr-ctc-en-int8-2025-12-25/model.int8.onnx" | ||
| let tokens = | ||
| "./sherpa-onnx-medasr-ctc-en-int8-2025-12-25/tokens.txt" | ||
|
|
||
| let medasr = sherpaOnnxOfflineMedAsrCtcModelConfig( | ||
| model: model | ||
| ) | ||
|
|
||
| let modelConfig = sherpaOnnxOfflineModelConfig( | ||
| tokens: tokens, | ||
| debug: 1, | ||
| medasr: medasr | ||
| ) | ||
|
|
||
| let featConfig = sherpaOnnxFeatureConfig() | ||
| var config = sherpaOnnxOfflineRecognizerConfig( | ||
| featConfig: featConfig, | ||
| modelConfig: modelConfig | ||
| ) | ||
|
|
||
| let recognizer = SherpaOnnxOfflineRecognizer(config: &config) | ||
|
|
||
| let filePath = "./sherpa-onnx-medasr-ctc-en-int8-2025-12-25/test_wavs/0.wav" | ||
| let audio = SherpaOnnxWaveWrapper.readWave(filename: filePath) | ||
|
|
||
| let result = recognizer.decode(samples: audio.samples, sampleRate: audio.sampleRate) | ||
| print("decode done") | ||
|
|
||
| print("\nresult is:\n\(result.text)") | ||
| if result.timestamps.count != 0 { | ||
| print("\ntimestamps is:\n\(result.timestamps)") | ||
| } | ||
| } |
There was a problem hiding this comment.
This function can be improved in a few ways for better maintainability and adherence to Swift idioms:
- Reduce Redundancy: The model directory path is repeated multiple times. It's better to define it as a constant and reuse it.
- Debug Flag: For consistency with other examples,
debugshould be set to0instead of1. - Idiomatic Check: Use
!result.timestamps.isEmptyinstead ofresult.timestamps.count != 0to check for an empty collection, which is more idiomatic in Swift.
Here is a suggested refactoring that applies these improvements.
func run() {
let modelDir = "./sherpa-onnx-medasr-ctc-en-int8-2025-12-25"
let model = "\(modelDir)/model.int8.onnx"
let tokens = "\(modelDir)/tokens.txt"
let medasr = sherpaOnnxOfflineMedAsrCtcModelConfig(
model: model
)
let modelConfig = sherpaOnnxOfflineModelConfig(
tokens: tokens,
debug: 0,
medasr: medasr
)
let featConfig = sherpaOnnxFeatureConfig()
var config = sherpaOnnxOfflineRecognizerConfig(
featConfig: featConfig,
modelConfig: modelConfig
)
let recognizer = SherpaOnnxOfflineRecognizer(config: &config)
let filePath = "\(modelDir)/test_wavs/0.wav"
let audio = SherpaOnnxWaveWrapper.readWave(filename: filePath)
let result = recognizer.decode(samples: audio.samples, sampleRate: audio.sampleRate)
print("decode done")
print("\nresult is:\n\(result.text)")
if !result.timestamps.isEmpty {
print("\ntimestamps is:\n\(result.timestamps)")
}
}| if [ ! -f ./sherpa-onnx-medasr-ctc-en-int8-2025-12-25/tokens.txt ]; then | ||
| curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-medasr-ctc-en-int8-2025-12-25.tar.bz2 | ||
| tar xvf sherpa-onnx-medasr-ctc-en-int8-2025-12-25.tar.bz2 | ||
| rm sherpa-onnx-medasr-ctc-en-int8-2025-12-25.tar.bz2 | ||
| fi |
There was a problem hiding this comment.
To improve readability and maintainability, you should define the model directory/archive name as a variable and reuse it. This avoids repeating the long string and makes it easier to update in the future.
| if [ ! -f ./sherpa-onnx-medasr-ctc-en-int8-2025-12-25/tokens.txt ]; then | |
| curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-medasr-ctc-en-int8-2025-12-25.tar.bz2 | |
| tar xvf sherpa-onnx-medasr-ctc-en-int8-2025-12-25.tar.bz2 | |
| rm sherpa-onnx-medasr-ctc-en-int8-2025-12-25.tar.bz2 | |
| fi | |
| model_name="sherpa-onnx-medasr-ctc-en-int8-2025-12-25" | |
| if [ ! -f ./${model_name}/tokens.txt ]; then | |
| curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/${model_name}.tar.bz2 | |
| tar xvf ${model_name}.tar.bz2 | |
| rm ${model_name}.tar.bz2 | |
| fi |
There was a problem hiding this comment.
Pull request overview
This PR adds Swift API support for the Google MedASR (Medical Automatic Speech Recognition) CTC model. The implementation follows the established pattern used for similar ASR models in the repository.
- Adds Swift wrapper functions for MedASR CTC model configuration
- Includes example implementation demonstrating model usage with test audio
- Integrates MedASR testing into the CI/CD pipeline
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| swift-api-examples/run-medasr-ctc-asr.sh | Shell script to download model files, compile, and run the MedASR CTC example |
| swift-api-examples/medasr-ctc.swift | Swift implementation demonstrating MedASR model usage for audio transcription |
| swift-api-examples/SherpaOnnx.swift | Adds MedASR model configuration wrapper functions to the Swift API |
| swift-api-examples/.gitignore | Adds medasr-ctc executable to the ignore list |
| .github/scripts/test-swift.sh | Integrates MedASR example into the CI test suite |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
csukuangfj
changed the title
Summary by CodeRabbit
New Features
Tests
✏️ Tip: You can customize this high-level summary in your review settings.