Fix WhisperProcessor divide-by-zero when single prompt is provided - #2068
Conversation
…) in addition to payload.prompts Agent-Logs-Url: https://github.com/microsoft/onnxruntime-genai/sessions/7fa715ae-22b5-4596-8f04-14b90c1ad9c5 Co-authored-by: skottmckay <979079+skottmckay@users.noreply.github.com>
|
@copilot all changes should have tests to validate. please add. |
Agent-Logs-Url: https://github.com/microsoft/onnxruntime-genai/sessions/db68d8c7-d1a3-43dd-bd23-cf9078f22343 Co-authored-by: skottmckay <979079+skottmckay@users.noreply.github.com>
Added |
There was a problem hiding this comment.
Pull request overview
Fixes a crash in the audio preprocessing path for Whisper models when MultiModalProcessor is invoked with a single prompt string. The core issue was WhisperProcessor::Process always calling Tokenizer::EncodeBatch with an empty prompt span, triggering a divide-by-zero in the tokenizer’s batching logic.
Changes:
- Update
WhisperProcessor::Processto prioritizepayload.prompts(batch) and fall back topayload.prompt(single prompt), otherwise throw a clearstd::runtime_error. - Add a Python regression test that calls the multimodal processor with a single prompt string and a single audio input.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
src/models/whisper_processor.cpp |
Avoids passing an empty prompt span into EncodeBatch by correctly handling the single-prompt payload field and emitting a descriptive error for missing prompts. |
test/python/test_onnxruntime_genai_api.py |
Adds coverage for the single-string prompt call path to prevent regressions of the divide-by-zero crash. |
WhisperProcessor::Processunconditionally calledtokenizer.EncodeBatch(payload.prompts), ignoringpayload.prompt. When invoked via the single-string overload ofMultiModalProcessor::Process,payload.promptsis an empty span, causing a divide-by-zero inEncodeBatch(encoded.size() / strings.size()).Changes
src/models/whisper_processor.cpp: Replace the unconditionalEncodeBatch(payload.prompts)call with a priority check:payload.promptsnon-empty → use as-is (batch path)payload.promptnon-empty → wrap in a single-elementconst char*span and pass toEncodeBatch(single-prompt path)std::runtime_errortest/python/test_onnxruntime_genai_api.py: Addedtest_audio_preprocessing_single_promptto validate the single-prompt code path. The test passes a plain string (not a list) to the processor with a single audio file, which exercises thepayload.promptbranch that previously caused the divide-by-zero.