Skip to content

support hotwords for FunASR model#39674

Open
AllenDou wants to merge 4 commits intovllm-project:mainfrom
AllenDou:asr_hotwords
Open

support hotwords for FunASR model#39674
AllenDou wants to merge 4 commits intovllm-project:mainfrom
AllenDou:asr_hotwords

Conversation

@AllenDou
Copy link
Copy Markdown
Contributor

@AllenDou AllenDou commented Apr 13, 2026

Hi @DarkLight1337 @Isotr0py this PR adds support for hotwords for ASR model such as FunASR, please take a look

server,
vllm serve allendou/Fun-ASR-Nano-2512-vllm -tp=2 --dtype=float32

client
python3 openai_transcription_client.py --repetition_penalty=1.0 --audio_path=/root/mary_had_lamb.wav --hotwords="hello,world"

related PR #33247

zixiao added 3 commits April 13, 2026 14:09
Signed-off-by: zixiao <shunli.dsl@alibaba-inc.com>
Signed-off-by: zixiao <shunli.dsl@alibaba-inc.com>
Signed-off-by: zixiao <shunli.dsl@alibaba-inc.com>
@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Apr 13, 2026

Documentation preview: https://vllm--39674.org.readthedocs.build/en/39674/

@mergify mergify bot added documentation Improvements or additions to documentation frontend qwen Related to Qwen models labels Apr 13, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces 'hotwords' support for speech-to-text transcription, updating the OpenAI-compatible entrypoint and multiple model executors. A review comment pointed out a potential AttributeError in the shared preprocessing function when handling translation requests that lack the 'hotwords' attribute, suggesting the use of getattr for safe access.

else None
)

hotwords = request.hotwords if request.hotwords else None
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Accessing request.hotwords directly will raise an AttributeError when the request is a TranslationRequest, as the hotwords field is currently only defined in TranscriptionRequest within protocol.py. Since _preprocess_speech_to_text is shared between transcription and translation tasks, you should use getattr to safely access this field.

Suggested change
hotwords = request.hotwords if request.hotwords else None
hotwords = getattr(request, "hotwords", None) or None

Signed-off-by: zixiao <shunli.dsl@alibaba-inc.com>
@DarkLight1337
Copy link
Copy Markdown
Member

Let's get #36268 merged first, then we can integrate this additional parameter

@AllenDou
Copy link
Copy Markdown
Contributor Author

Let's get #36268 merged first, then we can integrate this additional parameter

ok

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation frontend qwen Related to Qwen models

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants