Skip to content

Fix WhisperProcessor divide-by-zero when single prompt is provided - #2068

Merged
baijumeswani merged 3 commits into
mainfrom
copilot/update-whisper-processor-payload-handling
Apr 6, 2026
Merged

Fix WhisperProcessor divide-by-zero when single prompt is provided#2068
baijumeswani merged 3 commits into
mainfrom
copilot/update-whisper-processor-payload-handling

Conversation

Copilot AI commented Apr 5, 2026

Copy link
Copy Markdown
Contributor

WhisperProcessor::Process unconditionally called tokenizer.EncodeBatch(payload.prompts), ignoring payload.prompt. When invoked via the single-string overload of MultiModalProcessor::Process, payload.prompts is an empty span, causing a divide-by-zero in EncodeBatch (encoded.size() / strings.size()).

Changes

  • src/models/whisper_processor.cpp: Replace the unconditional EncodeBatch(payload.prompts) call with a priority check:
    1. payload.prompts non-empty → use as-is (batch path)
    2. payload.prompt non-empty → wrap in a single-element const char* span and pass to EncodeBatch (single-prompt path)
    3. Neither set → throw a descriptive std::runtime_error
std::shared_ptr<Tensor> input_ids;
if (!payload.prompts.empty()) {
  input_ids = tokenizer.EncodeBatch(payload.prompts);
} else if (!payload.prompt.empty()) {
  const char* prompt_ptr = payload.prompt.c_str();
  input_ids = tokenizer.EncodeBatch(std::span<const char*>(&prompt_ptr, 1));
} else {
  throw std::runtime_error("No prompt or prompts provided to WhisperProcessor::Process.");
}
  • test/python/test_onnxruntime_genai_api.py: Added test_audio_preprocessing_single_prompt to validate the single-prompt code path. The test passes a plain string (not a list) to the processor with a single audio file, which exercises the payload.prompt branch that previously caused the divide-by-zero.

…) in addition to payload.prompts

Agent-Logs-Url: https://github.com/microsoft/onnxruntime-genai/sessions/7fa715ae-22b5-4596-8f04-14b90c1ad9c5

Co-authored-by: skottmckay <979079+skottmckay@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix WhisperProcessor to check payload.prompt and payload.prompts Fix WhisperProcessor divide-by-zero when single prompt is provided Apr 5, 2026
Copilot AI requested a review from skottmckay April 5, 2026 07:10
@skottmckay

Copy link
Copy Markdown
Contributor

@copilot all changes should have tests to validate. please add.

Copilot AI commented Apr 5, 2026

Copy link
Copy Markdown
Contributor Author

@copilot all changes should have tests to validate. please add.

Added test_audio_preprocessing_single_prompt in test/python/test_onnxruntime_genai_api.py (commit 43256d3). It uses the same audio-preprocessing model and 1272-141231-0002.mp3 audio file as the existing tests, but passes a single string prompt (not a list) to the processor — which exercises the fixed payload.prompt code path that previously caused a divide-by-zero.

@skottmckay
skottmckay marked this pull request as ready for review April 5, 2026 08:10
Copilot AI review requested due to automatic review settings April 5, 2026 08:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a crash in the audio preprocessing path for Whisper models when MultiModalProcessor is invoked with a single prompt string. The core issue was WhisperProcessor::Process always calling Tokenizer::EncodeBatch with an empty prompt span, triggering a divide-by-zero in the tokenizer’s batching logic.

Changes:

  • Update WhisperProcessor::Process to prioritize payload.prompts (batch) and fall back to payload.prompt (single prompt), otherwise throw a clear std::runtime_error.
  • Add a Python regression test that calls the multimodal processor with a single prompt string and a single audio input.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
src/models/whisper_processor.cpp Avoids passing an empty prompt span into EncodeBatch by correctly handling the single-prompt payload field and emitting a descriptive error for missing prompts.
test/python/test_onnxruntime_genai_api.py Adds coverage for the single-string prompt call path to prevent regressions of the divide-by-zero crash.

@baijumeswani
baijumeswani enabled auto-merge (squash) April 6, 2026 16:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

WhisperProcessor::Process doesn't handle payload.prompt being set.

4 participants