Add Pascal API examples for ZipVoice models - #3351
Conversation
dosubot
Bot
added
the
size:L
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the Pascal API examples by integrating a new demonstration for ZipVoice Chinese/English zero-shot Text-to-Speech models. A key aspect of this change is the resolution of a complex cross-language compatibility issue concerning floating-point exception handling between C and Pascal on x64 systems, ensuring the TTS functionality operates robustly across different environments. Additionally, the request includes general improvements to the Pascal API's internal structure and memory management, alongside the introduction of progress reporting for TTS generation. Highlights
Ignored Files
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
|
Caution Review failedPull request was closed or merged during review 📝 WalkthroughWalkthroughThis PR introduces ZipVoice Chinese/English text-to-speech support to the Pascal API binding. It includes refactored callback type signatures in the core Pascal wrapper, a new ZipVoice example application with progress reporting, an automated build/run script, and supporting documentation and workflow updates. Changes
Sequence DiagramsequenceDiagram
participant Client as Pascal Client
participant ONNX as Sherpa ONNX<br/>(C Library)
participant Models as Model Files
participant Audio as Audio Output
Client->>Models: Load tokens, encoder,<br/>decoder, vocoder
Client->>ONNX: Create OfflineTtsConfig<br/>(with model paths)
Client->>ONNX: Build OfflineTts instance
Client->>Audio: Load reference audio
Client->>ONNX: Prepare GenerationConfig<br/>(reference text/audio)
Client->>ONNX: Call Generate with<br/>ProgressCallback
loop During Generation
ONNX->>Client: Invoke ProgressCallback<br/>(progress %)
Client->>Client: Print progress
end
ONNX->>Client: Return Generated Audio
Client->>Audio: Write WAV file
Client->>ONNX: Free TTS instance
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested labels
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
The pull request introduces a new Pascal example for ZipVoice Chinese/English zero-shot TTS, including a new run-zipvoice-zh-en.sh script and a zipvoice-zh-en.pas program. It also updates the pocket-en.pas example to use a progress callback. Significant improvements were made to the Pascal API bindings (sherpa_onnx.pas) by adding an initialization block to set the floating-point exception mask, which fixes EOverflow errors and ensures consistent behavior with the C API. Memory management for ReferenceAudio was made more robust by using GetMem and FreeMem within a try..finally block to prevent memory leaks. Additionally, redundant explicit pointer type aliases for callback functions were removed, improving type correctness and clarity in the API definitions.
| initialization | ||
| { Match the C API's default behavior. PocketTTS can raise FP overflow flags | ||
| during native inference on some platforms, and Free Pascal would otherwise | ||
| surface them as EOverflow. | ||
| See also https://github.com/k2-fsa/sherpa-onnx/pull/3351 | ||
| } |
There was a problem hiding this comment.
Adding the initialization block to set the floating-point exception mask is a critical fix. As detailed in the PR description, this addresses the cross-language floating-point exception policy mismatch on x64 platforms, preventing EOverflow errors and ensuring consistent behavior with the C API. This significantly improves the stability of the Pascal bindings.
| ReferenceAudio := GenerationConfig.ReferenceAudio; | ||
| CReferenceAudio := nil; | ||
| C.ReferenceAudio := nil; | ||
| C.ReferenceAudioLen := Length(ReferenceAudio); | ||
| if C.ReferenceAudioLen > 0 then | ||
| begin | ||
| GetMem(CReferenceAudio, C.ReferenceAudioLen * SizeOf(Single)); | ||
| Move(ReferenceAudio[0], CReferenceAudio[0], C.ReferenceAudioLen * SizeOf(Single)); | ||
| C.ReferenceAudio := CReferenceAudio; | ||
| end; | ||
| C.ReferenceSampleRate:= GenerationConfig.ReferenceSampleRate; | ||
| C.ReferenceText := PAnsiChar(GenerationConfig.ReferenceText); | ||
| ReferenceText := GenerationConfig.ReferenceText; | ||
| C.ReferenceText := PAnsiChar(ReferenceText); | ||
| C.NumSteps := GenerationConfig.NumSteps; | ||
| C.Extra := PAnsiChar(GenerationConfig.Extra); | ||
|
|
||
| Result := Default(TSherpaOnnxGeneratedAudio); | ||
|
|
||
| Audio := SherpaOnnxOfflineTtsGenerateWithConfig(Self.Handle, PAnsiChar(Text), | ||
| @C, Callback, Arg); | ||
| Extra := GenerationConfig.Extra; | ||
| C.Extra := PAnsiChar(Extra); | ||
|
|
||
| if Audio = nil then | ||
| Exit; | ||
|
|
||
| SetLength(Result.Samples, Audio^.N); | ||
| Result.SampleRate := Audio^.SampleRate; | ||
|
|
||
| if Audio^.N > 0 then | ||
| Move(Audio^.Samples[0], Result.Samples[0], Audio^.N * SizeOf(Single)); | ||
| Audio := nil; | ||
| try | ||
| Audio := SherpaOnnxOfflineTtsGenerateWithConfig(Self.Handle, PAnsiChar(Text), | ||
| @C, Callback, Arg); | ||
| finally | ||
| if CReferenceAudio <> nil then | ||
| FreeMem(CReferenceAudio); | ||
| end; |
There was a problem hiding this comment.
The handling of ReferenceAudio by allocating memory with GetMem and freeing it with FreeMem within a try..finally block is a robust improvement. This ensures proper memory management when converting Pascal's dynamic array (array of Single) to a C-compatible pointer (pcfloat) and prevents memory leaks, which is critical for stability.
|
|
||
| PSherpaOnnxGeneratedAudioCallbackWithArg = ^TSherpaOnnxGeneratedAudioCallbackWithArg; | ||
|
|
||
| TSherpaOnnxGeneratedAudioCallbackWithArg = function( |
There was a problem hiding this comment.
The removal of the explicit pointer type alias PSherpaOnnxGeneratedAudioCallbackWithArg is a good correction. In Free Pascal, function types are already treated as pointers when used in parameter lists for external functions, making the explicit pointer alias redundant and potentially misleading.
|
|
||
| PSherpaOnnxGeneratedAudioProgressCallbackWithArg = ^TSherpaOnnxGeneratedAudioProgressCallbackWithArg; | ||
|
|
||
| TSherpaOnnxGeneratedAudioProgressCallbackWithArg = function( |
There was a problem hiding this comment.
Similarly, removing the explicit pointer type alias PSherpaOnnxGeneratedAudioProgressCallbackWithArg is a correct and consistent change, aligning with Free Pascal's handling of function types.
| function Generate(Text: AnsiString; SpeakerId: Integer; | ||
| Speed: Single; | ||
| Callback: PSherpaOnnxGeneratedAudioCallbackWithArg; | ||
| Callback: TSherpaOnnxGeneratedAudioCallbackWithArg; |
There was a problem hiding this comment.
Changing the Callback parameter type from PSherpaOnnxGeneratedAudioCallbackWithArg to TSherpaOnnxGeneratedAudioCallbackWithArg directly uses the function type, which is the correct approach for passing callback functions in Free Pascal to C-style APIs. This aligns with the removal of the redundant pointer type alias.
| function Generate(Text: AnsiString; | ||
| GenerationConfig: TSherpaOnnxGenerationConfig; | ||
| Callback: PSherpaOnnxGeneratedAudioProgressCallbackWithArg; | ||
| Callback: TSherpaOnnxGeneratedAudioProgressCallbackWithArg; |
There was a problem hiding this comment.
This change correctly updates the Callback parameter type to TSherpaOnnxGeneratedAudioProgressCallbackWithArg, ensuring consistency with Free Pascal's function pointer handling and the previous type alias removal.
| implementation | ||
|
|
||
| uses | ||
| Math, |
There was a problem hiding this comment.
Adding Math to the uses clause is necessary to use SetExceptionMask, which is crucial for handling floating-point exceptions as described in the PR body. This ensures the required functionality is available.
| end; | ||
|
|
||
| PSherpaOnnxGenerationConfig = ^TSherpaOnnxGenerationConfig; | ||
| PSherpaOnnxGenerationConfig = ^SherpaOnnxGenerationConfig; |
There was a problem hiding this comment.
Correcting the pointer type alias from ^TSherpaOnnxGenerationConfig to ^SherpaOnnxGenerationConfig is important for type correctness. SherpaOnnxGenerationConfig is the record type, and the pointer should refer to it directly.
| function SherpaOnnxOfflineTtsGenerateWithCallbackWithArg(Tts: Pointer; | ||
| Text: PAnsiChar; Sid: cint32; Speed: cfloat; | ||
| Callback: PSherpaOnnxGeneratedAudioCallbackWithArg; | ||
| Callback: TSherpaOnnxGeneratedAudioCallbackWithArg; |
There was a problem hiding this comment.
Updating the Callback parameter type to TSherpaOnnxGeneratedAudioCallbackWithArg for the external function declaration ensures that the C API correctly receives the Pascal function pointer. This is consistent with the other callback type corrections.
| function SherpaOnnxOfflineTtsGenerateWithConfig(Tts: Pointer; | ||
| Text: PAnsiChar; config: PSherpaOnnxGenerationConfig; | ||
| Callback: PSherpaOnnxGeneratedAudioProgressCallbackWithArg; | ||
| Callback: TSherpaOnnxGeneratedAudioProgressCallbackWithArg; |
There was a problem hiding this comment.
This change correctly updates the Callback parameter type to TSherpaOnnxGeneratedAudioProgressCallbackWithArg for the external function, maintaining consistency across the API definitions.
There was a problem hiding this comment.
Pull request overview
Adds a Pascal ZipVoice (zh-en) TTS example and updates the Pascal bindings to better interoperate with the C API during inference (notably around floating-point exception handling).
Changes:
- Add ZipVoice zh-en Pascal example + runner script and integrate it into CI.
- Refactor OfflineTTS audio extraction and update callback typing in Pascal bindings.
- Mask floating-point exceptions in the Pascal unit initialization to avoid
EOverflowduring native inference.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| sherpa-onnx/pascal-api/sherpa_onnx.pas | Refactors TTS generation plumbing, adjusts callback types, copies reference audio for C interop, and masks FP exceptions at init. |
| pascal-api-examples/tts/zipvoice-zh-en.pas | New ZipVoice Chinese/English zero-shot TTS example using progress callback and reference audio. |
| pascal-api-examples/tts/run-zipvoice-zh-en.sh | Builds/links shared lib, downloads ZipVoice + vocoder assets, compiles and runs the new example. |
| pascal-api-examples/tts/pocket-en.pas | Adds a progress callback and wires it into generation. |
| pascal-api-examples/tts/README.md | Documents the new ZipVoice example runner. |
| pascal-api-examples/tts/.gitignore | Ignores the new ZipVoice example binary. |
| .github/workflows/pascal.yaml | Runs the new ZipVoice example in the Pascal workflow and cleans up artifacts. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| ReferenceAudio := GenerationConfig.ReferenceAudio; | ||
| CReferenceAudio := nil; | ||
| C.ReferenceAudio := nil; | ||
| C.ReferenceAudioLen := Length(ReferenceAudio); | ||
| if C.ReferenceAudioLen > 0 then | ||
| begin | ||
| GetMem(CReferenceAudio, C.ReferenceAudioLen * SizeOf(Single)); | ||
| Move(ReferenceAudio[0], CReferenceAudio[0], C.ReferenceAudioLen * SizeOf(Single)); | ||
| C.ReferenceAudio := CReferenceAudio; | ||
| end; |
| initialization | ||
| { Match the C API's default behavior. PocketTTS can raise FP overflow flags | ||
| during native inference on some platforms, and Free Pascal would otherwise | ||
| surface them as EOverflow. | ||
| See also https://github.com/k2-fsa/sherpa-onnx/pull/3351 | ||
| } | ||
| SetExceptionMask([exInvalidOp, exDenormalized, exZeroDivide, exOverflow, | ||
| exUnderflow, exPrecision]); |
| SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) | ||
| SHERPA_ONNX_DIR=$(cd $SCRIPT_DIR/../.. && pwd) |
|
|
||
| echo "SHERPA_ONNX_DIR: $SHERPA_ONNX_DIR" | ||
|
|
||
| if [[ ! -f ../../build/install/lib/libsherpa-onnx-c-api.dylib && ! -f ../../build/install/lib/libsherpa-onnx-c-api.so && ! -f ../../build/install/lib/sherpa-onnx-c-api.dll ]]; then |
| fpc \ | ||
| -dSHERPA_ONNX_USE_SHARED_LIBS \ | ||
| -Fu$SHERPA_ONNX_DIR/sherpa-onnx/pascal-api \ | ||
| -Fl$SHERPA_ONNX_DIR/build/install/lib \ | ||
| ./zipvoice-zh-en.pas |
https://github.com/k2-fsa/sherpa-onnx/actions/runs/23187281508/job/67373909595
From ChatGPT:
What you’re seeing is a classic cross-language floating-point exception policy mismatch—and your symptoms match it almost perfectly:
EOverflow: Floating point overflow(Pascal exception)🔍 What’s really happening
You are mixing:
Inf/NaN)EOverflow,EDivByZero, etc.)👉 On x64 platforms, the floating-point unit (SSE/AVX via MXCSR register) is strictly used and shared globally.
👉 On ARM64:
On x64:
Pascal runtime sets something like:
SetExceptionMask([]); // unmask → raise exceptionsYour C code (or a library) does something like:
👉 In C mindset: OK → returns
Inf👉 In Pascal mindset: ❌ raises
EOverflow💥 Result: crash in Pascal even though C thinks it’s fine
🧠 The deeper issue: shared FPU state
The floating-point control state (MXCSR on x86_64) includes:
👉 This state is:
So:
🔬 Why ARM64 behaves differently
On ARM64 (Ubuntu):
Inf, no exception👉 That’s why your code “works” there
💥 Typical triggers
You’ll see this with:
exp,pow)sqrt(-1)in real domain)✅ Fix strategies
🔧 Option 1: Make Pascal behave like C (most common fix)
Mask FP exceptions in Pascal:
uses Math; SetExceptionMask([ exInvalidOp, exDenormalized, exZeroDivide, exOverflow, exUnderflow, exPrecision ]);👉 This makes Pascal:
Inf/NaNEOverflow🔧 Option 2: Temporarily mask around C calls
If you want Pascal strictness elsewhere:
🔧 Option 3: Fix on the C side (less reliable)
You can try to enforce masking in C:
🔧 Option 4: Detect and clamp values (robust approach)
Before risky operations:
Or check results:
🧪 Debug tip
Enable exceptions deliberately to find the culprit:
SetExceptionMask([]); // unmask everything👉 Then you’ll see exactly:
🧩 Key takeaway
This is not a “bug in your math”—it’s a policy conflict:
🧠 One-line summary
If you want, I can help you pinpoint the exact line causing the overflow (e.g., by instrumenting your code or analyzing a snippet).
Summary by CodeRabbit
New Features
Documentation
Improvements