Whisper ASR Model Support for MergeKit

[sachaarbonel]

Whisper ASR Model Support for MergeKit

This PR adds comprehensive support for merging OpenAI's Whisper automatic speech recognition (ASR) models in MergeKit.

Key Features

• Whisper Architecture Support: Added architecture definition for Whisper encoder-decoder models
• Encoder-Decoder Weighted Merge Method: Implemented a new merge method that allows applying different weights to encoder (audio processing) and decoder (text generation) components
• Whisper LoRA Support: Added tools for extracting and merging LoRA adapters from fine-tuned Whisper models
• Documentation & Examples: Created a detailed guide for merging Whisper models with various example configurations

Use Cases

This implementation enables several powerful workflows:

• Merging multilingual Whisper models while preserving capabilities across languages
• Combining models where one has better audio processing and another has better text generation
• Merging LoRA fine-tuned Whisper models for specialized domains or languages

Testing

Added comprehensive tests for Whisper model merging functionality, including specific tests for the new encoder-decoder weighted merge method.

Future Work

• Add support for more specialized Whisper merging techniques
• Implement evaluation metrics specific to ASR models

[github-actions]

Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.

---

I have read the CLA Document and I hereby sign the CLA

---

_{You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.}