Fix quantization in Whisper model export

[jiafatom]

Description

Fix quantization in Whisper model export

Motivation and Context

As titled.

[github-actions]

You can commit the suggested changes from lintrunner.

[github-actions]

You can commit the suggested changes from lintrunner.

[github-actions]

You can commit the suggested changes from lintrunner.

[kunal-vaishnavi]

Since the big models CI pipeline was deleted, we should verify that these commands all still work.

[jiafatom]

Since the big models CI pipeline was deleted, we should verify that these commands all still work.

I verified a lot of cases work. The issue is --model_impl openai case.
When we use
python3 -m models.whisper.convert_to_onnx -m openai/whisper-tiny --model_impl openai --output wtiny-fp32-cpu-oai --precision fp32 --provider cpu --overwrite --use_external_data_format --optimize_onnx --no_beam_search_op --output_cross_qk ;
we will see
TypeError: scaled_dot_product_attention(): argument 'is_causal' must be bool, not Tensor

However, I see the opeai-whisper bug mentioned in the script

       # Run forward pass
       # NOTE: There is a bug with openai-whisper==20240930 with the introduction of SDPA.
       # In the Whisper codebase, the following line
       #
       # is_causal = mask is not None and n_ctx > 1
       #
       # has been added where `mask` is a torch tensor. The right-hand side evaluates to `tensor(True/False)`
       # but `is_causal` only accepts the boolean value. The fix is to apply `.item()` after the right-hand
       # side has been evaluated. In other words, the line should be
       #
       # is_causal = (mask is not None and n_ctx > 1).item()
       #
       # instead.

[kunal-vaishnavi]

Since the big models CI pipeline was deleted, we should verify that these commands all still work.

I verified a lot of cases work. The issue is --model_impl openai case. When we use python3 -m models.whisper.convert_to_onnx -m openai/whisper-tiny --model_impl openai --output wtiny-fp32-cpu-oai --precision fp32 --provider cpu --overwrite --use_external_data_format --optimize_onnx --no_beam_search_op --output_cross_qk ; we will see TypeError: scaled_dot_product_attention(): argument 'is_causal' must be bool, not Tensor

However, I see the opeai-whisper bug mentioned in the script

       # Run forward pass
       # NOTE: There is a bug with openai-whisper==20240930 with the introduction of SDPA.
       # In the Whisper codebase, the following line
       #
       # is_causal = mask is not None and n_ctx > 1
       #
       # has been added where `mask` is a torch tensor. The right-hand side evaluates to `tensor(True/False)`
       # but `is_causal` only accepts the boolean value. The fix is to apply `.item()` after the right-hand
       # side has been evaluated. In other words, the line should be
       #
       # is_causal = (mask is not None and n_ctx > 1).item()
       #
       # instead.

For the --model_impl openai case, can you verify it works with the version of openai-whisper in requirements.txt?

onnxruntime/onnxruntime/python/tools/transformers/models/whisper/requirements.txt

Line 3 in 7cc28b0

openai-whisper==20240927

Separately, an internal customer mentioned there were export issues with torch==2.9.0 while export succeeded with torch==2.7.0.

onnxruntime/onnxruntime/python/tools/transformers/models/whisper/requirements.txt

Line 1 in 7cc28b0

torch>=2.7.0

Let's pin the torch version in requirements.txt to 2.7.0.

[jiafatom]

python3 -m models.whisper.convert_to_onnx -m openai/whisper-tiny --model_impl openai --output wtiny-fp32-cpu-oai --precision fp32 --provider cpu --overwrite --use_external_data_format --optimize_onnx --no_beam_search_op --output_cross_qk

Tested with openai-whisper==20240927 and it works!

[github-actions]

You can commit the suggested changes from lintrunner.

[github-actions]

You can commit the suggested changes from lintrunner.