Compatibility Issue with openai_whisper-large-v3-v20240930_turbo_632MB Model on iOS 16.6

[ppcfan]

In Package.swift, the required iOS version is set to 16. However, I encountered an error when attempting to load the openai_whisper-large-v3-v20240930_turbo_632MB model on iOS 16.6. It seems the issue is related to the iOS version. When I tried loading the model on an iOS 17.5.1 system, it worked without any problems. Does this mean I need to upgrade my iOS to version 17 or above to use this model? Thank you!

======================================================================
Selected Model: nil
Computing Options:
- Mel Spectrogram: cpuAndGPU
- Audio Encoder: cpuAndNeuralEngine
- Text Decoder: cpuAndNeuralEngine
- Prefill Data: cpuOnly
2025-01-04 23:25:15.037773-0800 testapp[579:68923] [WhisperKit] Searching for models matching "openai_whisper-large-v3-v20240930_turbo_632MB/" in Repo(id: "argmaxinc/whisperkit-coreml", type: Hub.Hub.RepoType.models)
2025-01-04 23:25:15.263163-0800 testapp[579:68919] [WhisperKit] Downloading model openai_whisper-large-v3-v20240930_turbo_632MB...
2025-01-04 23:25:15.471312-0800 testapp[579:68943] [WhisperKit] <NSProgress: 0x283eb4880> : Parent: 0x0 (portion: 0) / Fraction completed: 1.0000 / Completed: 22 of 22
2025-01-04 23:25:15.471729-0800 testapp[579:68943] [WhisperKit] Loading models from /var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB with prewarmMode: true
2025-01-04 23:25:15.473444-0800 testapp[579:68943] [WhisperKit] Loading feature extractor
2025-01-04 23:25:15.497465-0800 testapp[579:68943] Metal API Validation Enabled
2025-01-04 23:25:15.683126-0800 testapp[579:68943] [WhisperKit] Loaded feature extractor
2025-01-04 23:25:15.683199-0800 testapp[579:68943] [WhisperKit] Loading text decoder prefill data
2025-01-04 23:25:15.693728-0800 testapp[579:68919] [coreml] MLModelAsset: load failed with error Error Domain=com.apple.CoreML Code=70 "at /private/var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB/TextDecoderContextPrefill.mlmodelc/model.mil:4:4: Validation error parsing MIL model: at /private/var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB/TextDecoderContextPrefill.mlmodelc/model.mil:4:4: Specialization 'ios17' not in blocks." UserInfo={NSLocalizedDescription=at /private/var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB/TextDecoderContextPrefill.mlmodelc/model.mil:4:4: Validation error parsing MIL model: at /private/var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB/TextDecoderContextPrefill.mlmodelc/model.mil:4:4: Specialization 'ios17' not in blocks., NSUnderlyingError=0x281f38090 {Error Domain=com.apple.CoreML Code=203 "(null)"}}
2025-01-04 23:25:15.693938-0800 testapp[579:68919] [coreml] MLModelAsset: modelWithError: load failed with error Error Domain=com.apple.CoreML Code=70 "at /private/var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB/TextDecoderContextPrefill.mlmodelc/model.mil:4:4: Validation error parsing MIL model: at /private/var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB/TextDecoderContextPrefill.mlmodelc/model.mil:4:4: Specialization 'ios17' not in blocks." UserInfo={NSLocalizedDescription=at /private/var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB/TextDecoderContextPrefill.mlmodelc/model.mil:4:4: Validation error parsing MIL model: at /private/var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB/TextDecoderContextPrefill.mlmodelc/model.mil:4:4: Specialization 'ios17' not in blocks., NSUnderlyingError=0x281f38090 {Error Domain=com.apple.CoreML Code=203 "(null)"}}
Error prewarming models, retrying: at /private/var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB/TextDecoderContextPrefill.mlmodelc/model.mil:4:4: Validation error parsing MIL model: at /private/var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB/TextDecoderContextPrefill.mlmodelc/model.mil:4:4: Specialization 'ios17' not in blocks.
Selected Model: nil
Computing Options:
- Mel Spectrogram: cpuAndGPU
- Audio Encoder: cpuAndNeuralEngine
- Text Decoder: cpuAndNeuralEngine
- Prefill Data: cpuOnly
2025-01-04 23:25:15.694482-0800 testapp[579:68943] [WhisperKit] Searching for models matching "openai_whisper-large-v3-v20240930_turbo_632MB/" in Repo(id: "argmaxinc/whisperkit-coreml", type: Hub.Hub.RepoType.models)
2025-01-04 23:25:15.872066-0800 testapp[579:68923] [WhisperKit] Downloading model openai_whisper-large-v3-v20240930_turbo_632MB...
2025-01-04 23:25:16.082714-0800 testapp[579:68918] [WhisperKit] <NSProgress: 0x283eb0a00> : Parent: 0x0 (portion: 0) / Fraction completed: 1.0000 / Completed: 22 of 22
2025-01-04 23:25:16.083012-0800 testapp[579:68918] [WhisperKit] Loading models from /var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB with prewarmMode: true
2025-01-04 23:25:16.084090-0800 testapp[579:68918] [WhisperKit] Loading feature extractor
2025-01-04 23:25:16.138115-0800 testapp[579:68919] [WhisperKit] Loaded feature extractor
2025-01-04 23:25:16.138183-0800 testapp[579:68919] [WhisperKit] Loading text decoder prefill data
2025-01-04 23:25:16.147573-0800 testapp[579:68919] [coreml] MLModelAsset: load failed with error Error Domain=com.apple.CoreML Code=70 "at /private/var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB/TextDecoderContextPrefill.mlmodelc/model.mil:4:4: Validation error parsing MIL model: at /private/var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB/TextDecoderContextPrefill.mlmodelc/model.mil:4:4: Specialization 'ios17' not in blocks." UserInfo={NSLocalizedDescription=at /private/var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB/TextDecoderContextPrefill.mlmodelc/model.mil:4:4: Validation error parsing MIL model: at /private/var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB/TextDecoderContextPrefill.mlmodelc/model.mil:4:4: Specialization 'ios17' not in blocks., NSUnderlyingError=0x281f80540 {Error Domain=com.apple.CoreML Code=203 "(null)"}}
2025-01-04 23:25:16.147823-0800 testapp[579:68919] [coreml] MLModelAsset: modelWithError: load failed with error Error Domain=com.apple.CoreML Code=70 "at /private/var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB/TextDecoderContextPrefill.mlmodelc/model.mil:4:4: Validation error parsing MIL model: at /private/var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB/TextDecoderContextPrefill.mlmodelc/model.mil:4:4: Specialization 'ios17' not in blocks." UserInfo={NSLocalizedDescription=at /private/var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB/TextDecoderContextPrefill.mlmodelc/model.mil:4:4: Validation error parsing MIL model: at /private/var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB/TextDecoderContextPrefill.mlmodelc/model.mil:4:4: Specialization 'ios17' not in blocks., NSUnderlyingError=0x281f80540 {Error Domain=com.apple.CoreML Code=203 "(null)"}}
Error prewarming models, retrying: at /private/var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB/TextDecoderContextPrefill.mlmodelc/model.mil:4:4: Validation error parsing MIL model: at /private/var/mobile/Containers/Data/Application/D766C997-9DB8-48D7-A325-F1268D60D3C3/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-large-v3-v20240930_turbo_632MB/TextDecoderContextPrefill.mlmodelc/model.mil:4:4: Specialization 'ios17' not in blocks.

[atiorh]

Hello, could you try deleting TextDecoderContextPrefill.mlmodelc (This model is optional) and report back? The specific error context does not yet make sense to me.

[ppcfan]

Hello, could you try deleting TextDecoderContextPrefill.mlmodelc (This model is optional) and report back? The specific error context does not yet make sense to me.

Thank you for your response. After deleting the TextDecoderContextPrefill.mlmodelc file, the model can now continue loading. However, on an iPhone 12 running iOS 16.6.1, the entire loading process still cannot be completed, as the program crashes due to insufficient memory. I believe this might be a different issue.

I have a total of three devices: an iPhone 13 (iOS 17.5.1), an iPhone 12 (iOS 16.6.1), and an iPhone 11 (iOS 18.1). Using the same code, the iPhone 13 successfully completes the loading process, while both the iPhone 12 and iPhone 11 crash due to insufficient memory during the "loading audio encoder" step. Therefore, I suspect that this crash is caused by hardware limitations in combination with the system version.

I tried changing the audioEncoderCompute setting to .cpuAndGPU, but it still couldn’t load.

Thank you again for your assistance.

[atiorh]

Thank you for your response. After deleting the TextDecoderContextPrefill.mlmodelc file, the model can now continue loading.

Great! This is still a mystery we should track but I am glad you are unblocked for now.

However, on an iPhone 12 running iOS 16.6.1, the entire loading process still cannot be completed

Please refer to this device support map:
https://huggingface.co/argmaxinc/whisperkit-coreml/blob/main/config.json

or this live one from our regression test results:
https://huggingface.co/spaces/argmaxinc/whisperkit-benchmarks

TL;DR

• openai_whisper-large-v3-v20240930_turbo_632MB is good for iPhone 13 and newer
• openai_whisper-large-v3-v20240930_626MB is good for iPhone 12
• They are the "same" model but the latter removes a single optimization to enable iPhone 12
• We are working to make this automatic so you shouldn't have deal with this. Will land before 1.0!

Please report anything else you observe!

[atiorh]

@ppcfan Please confirm whenever you are able to test my recommendation

[ppcfan]

@ppcfan Please confirm whenever you are able to test my recommendation

Hi, @atiorh thank you for your recommendation. I tested the openai_whisper-large-v3-v20240930_626MB on an iPhone 12, but it still couldn't load due to insufficient memory. Therefore, I’ve kept the device restriction in my app set to iPhone 13 and above.