support hotwords for FunASR model#39674
Conversation
|
Documentation preview: https://vllm--39674.org.readthedocs.build/en/39674/ |
There was a problem hiding this comment.
Code Review
This pull request introduces 'hotwords' support for speech-to-text transcription, updating the OpenAI-compatible entrypoint and multiple model executors. A review comment pointed out a potential AttributeError in the shared preprocessing function when handling translation requests that lack the 'hotwords' attribute, suggesting the use of getattr for safe access.
| else None | ||
| ) | ||
|
|
||
| hotwords = request.hotwords if request.hotwords else None |
There was a problem hiding this comment.
Accessing request.hotwords directly will raise an AttributeError when the request is a TranslationRequest, as the hotwords field is currently only defined in TranscriptionRequest within protocol.py. Since _preprocess_speech_to_text is shared between transcription and translation tasks, you should use getattr to safely access this field.
| hotwords = request.hotwords if request.hotwords else None | |
| hotwords = getattr(request, "hotwords", None) or None |
|
Let's get #36268 merged first, then we can integrate this additional parameter |
ok |
Hi @DarkLight1337 @Isotr0py this PR adds support for hotwords for ASR model such as FunASR, please take a look
server,
vllm serve allendou/Fun-ASR-Nano-2512-vllm -tp=2 --dtype=float32client
python3 openai_transcription_client.py --repetition_penalty=1.0 --audio_path=/root/mary_had_lamb.wav --hotwords="hello,world"related PR #33247