Skip to content

fix(stt): wire stt.local.initial_prompt through to faster-whisper - #54202

Closed
Akloenx123 wants to merge 1 commit into
NousResearch:mainfrom
Akloenx123:fix/stt-local-initial-prompt
Closed

fix(stt): wire stt.local.initial_prompt through to faster-whisper#54202
Akloenx123 wants to merge 1 commit into
NousResearch:mainfrom
Akloenx123:fix/stt-local-initial-prompt

Conversation

@Akloenx123

Copy link
Copy Markdown

Summary

faster-whisper exposes an initial_prompt kwarg that biases the model's
vocabulary toward domain-specific terms — useful for transcribing voice notes
that contain product names, technical jargon, or other words the default model
rarely sees in training (e.g. "DeepSeek", "MiMo", "Kicktipp", "Cronjobs").

Before this change, only stt.local.language was wired through from
config.yaml to the transcribe() call. initial_prompt was silently
ignored even when set, forcing users to either patch the code or accept
garbled transcripts of technical vocabulary.

This commit reads stt.local.initial_prompt from config.yaml and passes
it as the initial_prompt kwarg to faster-whisper's transcribe(). The
setting is optional and defaults to absent — existing configs are unaffected.

Reproduction (before)

# config.yaml
stt:
  provider: local
  local:
    model: large-v3
    language: de
    initial_prompt: "DeepSeek, MiMo, MiniMax, Kicktipp, Cronjobs"

User sends a voice note saying "schick mir den DeepSeek V4 Pro Tipp".
Transcribed result: "schick mir den Debsig V4 Pro Tipp".
The initial_prompt line in config is silently ignored.

After this fix, the same voice note transcribes correctly: "schick mir den
DeepSeek V4 Pro Tipp".

Why this should merge

  1. Minimal blast radius — 8 lines of core code, all additive.
  2. Backwards compatibleinitial_prompt defaults to absent; existing
    configs behave identically.
  3. Solves a real reported problem — voice-to-text for technical vocab
    (model names, product names, internal jargon) is a recurring pain point.
  4. Mirrors existing language plumbing — same config-yml pattern,
    same read order, same opt-in shape.
  5. Tests included — three contracts: prompt set / unset / empty string.

Test plan

  • tests/tools/test_stt_initial_prompt.py — 3 tests, all passing
  • Existing tests/tools/test_transcription.py — unchanged behavior
  • Manual E2E with large-v3 and a domain-specific voice note:
    • Before: "Debsig", "Kick-Tip", "Conjobs"
    • After: "DeepSeek", "Kicktipp", "Cronjobs"

AGENTS.md alignment

This change follows the project's "narrow waist" guidance:

  • No new core tool. Uses existing stt.local config namespace.
  • No new HERMES_* env var. All wiring is config.yaml-only, per the
    "reject PRs that tell users to set X in your .env" rule.
  • Extend, don't duplicate. Reuses the existing _load_stt_config() /
    _forced_lang / transcribe_kwargs plumbing — no parallel code path.
  • Behavior contract tests, not snapshots. Tests assert the kwarg-dict
    invariant (initial_prompt is present iff config sets it), not frozen
    model output.

Happy to address review feedback or split this into smaller commits if
preferred.

faster-whisper exposes an `initial_prompt` kwarg that biases the model's
vocabulary toward domain-specific terms — useful for transcribing voice
notes that contain product names, technical jargon, or other words the
default model rarely sees in training (e.g. "DeepSeek", "MiMo",
"Kicktipp", "Cronjobs").

Before this change, only `stt.local.language` was wired through from
config.yaml to the transcribe() call. `initial_prompt` was silently
ignored even when set, forcing users to either patch the code or accept
garbled transcripts.

This commit reads `stt.local.initial_prompt` from config.yaml and passes
it as the `initial_prompt` kwarg. The setting is optional and defaults
to absent — existing configs are unaffected.

Tests cover three contracts: prompt set (passed through), prompt unset
(omitted), empty string (treated as unset). All three pass.
@alt-glitch alt-glitch added type/feature New feature or request tool/tts Text-to-speech and transcription P3 Low — cosmetic, nice to have duplicate This issue or pull request already exists labels Jun 28, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of #50684 — same mechanism: both wire stt.local.initial_prompt from config.yaml through to faster-whisper's transcribe() in tools/transcription_tools.py to bias vocabulary toward domain-specific terms, both targeting #50181. #50684 is the earlier open PR. Cross-linking so a maintainer can pick the canonical one.

@Akloenx123

Copy link
Copy Markdown
Author

Closing in favor of #50684 — opened earlier (2026-06-22 vs 2026-06-28), smaller diff (+50 vs +118), includes the cli-config.yaml.example doc update I missed, and handles the whitespace edge case (initial_prompt.strip()) my patch skipped.

Thanks to @BlackishGreen33 for getting there first. Marking as superseded so a maintainer only has to review one PR.

@Akloenx123 Akloenx123 closed this Jun 28, 2026
@Akloenx123
Akloenx123 deleted the fix/stt-local-initial-prompt branch June 28, 2026 11:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

duplicate This issue or pull request already exists P3 Low — cosmetic, nice to have tool/tts Text-to-speech and transcription type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants