feat(ffmpeg-whisper): add provider selection and gpu auto-detect

[POWERFULMOVES]

Summary

• add multi-provider transcription pipeline with GPU detection, faster-whisper default, and optional Qwen2 Audio support
• allow /yt/transcript and /yt/ingest to forward the selected transcription provider and document the new options
• extend ffmpeg-whisper unit tests to cover provider forwarding and error handling

Testing

• pytest pmoves/tests/test_ffmpeg_whisper.py

---

https://chatgpt.com/codex/tasks/task_b_68ce2ec34a9c8324ab794101b7d74e80

Summary by CodeRabbit

• New Features

  • Choose the transcription provider via a request parameter (default: faster-whisper; options: whisper, qwen2-audio). Provider flows from ingest to transcript.
  • Transcription responses now include provider, device, and model details.

• Improvements

  • Auto-detects CUDA GPUs and selects optimal compute type; falls back to CPU INT8 when needed.
  • Configurable audio processing timeout.

• Documentation

  • Expanded guidance on provider selection, defaults, GPU/diarization recommendations, and configuration options.

• Tests

  • Added coverage for provider overrides and invalid provider errors.

• Chores

  • Added dependencies to support new providers.

[coderabbitai]

Walkthrough

Multi-provider transcription support is added across docs, service code, and tests. The ffmpeg-whisper server now routes to faster-whisper, whisper, or qwen2-audio, with device/compute-type detection, configurable timeouts, and new dependencies. pmoves-yt propagates an optional provider field. Tests cover provider override, invalid provider handling, and payload propagation.

Changes

Cohort / File(s)	Summary
Docs: transcript provider semantics pmoves/docs/PMOVES.yt/PMOVES_YT.md	Documents provider parameter for transcript/ingest, default/provider options (faster-whisper, whisper, qwen2-audio), CUDA auto-detect, CPU fallback, and expanded configuration notes.
Dependencies for new backends pmoves/services/ffmpeg-whisper/requirements.txt	Adds faster-whisper==1.0.2, transformers==4.44.2, sentencepiece==0.2.0 for new provider support.
ffmpeg-whisper: multi-provider refactor pmoves/services/ffmpeg-whisper/server.py	Introduces provider routing, device/compute-type detection, configurable timeout, new model loaders/runners for faster-whisper and qwen2-audio, revised ffmpeg extraction, iterable segment handling, health annotations, and error guards for unsupported providers. Updates app metadata and response fields.
YouTube ingest/transcript provider propagation pmoves/services/pmoves-yt/yt.py	Extends payloads to include optional provider; restructures yt_ingest to build and pass a transcript payload with provider when present.
Tests for provider flow and validation pmoves/tests/test_ffmpeg_whisper.py	Adds tests for provider override and invalid provider (400). Updates fallback test to assert provider propagation in forwarded payload and response.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor U as Client
  participant YT as pmoves-yt
  participant FW as ffmpeg-whisper
  participant FWf as Provider Runner

  U->>YT: POST /yt/transcript {language, whisper_model?, provider?}
  YT->>FW: POST /transcribe {language, model, provider?}
  FW->>FW: Validate provider ∈ {faster-whisper, whisper, qwen2-audio}
  alt Invalid provider
    FW-->>U: 400 {"error":"unsupported provider"}
  else Valid provider
    FW->>FW: Select device (cuda/cpu) + compute_type
    FW->>FWf: Load model (cached) for provider
    FWf-->>FW: Model ready
    FW->>FWf: Transcribe(audio)
    FWf-->>FW: {segments, text, meta}
    FW-->>U: 200 {provider, model, device, segments}
  end

Loading

sequenceDiagram
  autonumber
  actor U as Client
  participant YT as pmoves-yt (ingest)
  participant YT2 as pmoves-yt (transcript)
  participant FW as ffmpeg-whisper

  U->>YT: POST /yt/ingest {video_id, namespace, ..., provider?}
  YT->>YT2: Build tr_payload (include provider if present)
  YT2->>FW: POST /transcribe tr_payload
  FW-->>YT2: Transcript result
  YT2-->>YT: Pass-through result
  YT-->>U: Response

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Poem

I twitched my ears at choices three,
Whisper winds, Qwen’s sea, and faster-whisper’s spree.
GPUs hum, CPUs sigh—still, captions bloom for me.
I nibble logs, hop through tests, provider set with glee.
Transcript trails of carrot tales—ship it, bun-degree! 🥕🐇

---

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Free

📥 Commits

Reviewing files that changed from the base of the PR and between 2689925 and 5160f8f.

📒 Files selected for processing (5)

• pmoves/docs/PMOVES.yt/PMOVES_YT.md (2 hunks)
• pmoves/services/ffmpeg-whisper/requirements.txt (1 hunks)
• pmoves/services/ffmpeg-whisper/server.py (9 hunks)
• pmoves/services/pmoves-yt/yt.py (2 hunks)
• pmoves/tests/test_ffmpeg_whisper.py (2 hunks)

---

Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login.

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[chatgpt-codex-connector]

Codex Review: Here are some suggestions.

Reply with @codex fix comments to fix any unresolved comments.

About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you open a pull request for review, mark a draft as ready, or comment "@codex review". If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex fix this CI failure" or "@codex address that feedback".

[chatgpt-codex-connector]

[P1] Use Qwen2 default model when provider is qwen2-audio

Selecting provider="qwen2-audio" without explicitly passing whisper_model currently uses DEFAULT_WHISPER_MODEL (large-v3) because model_name is computed before the provider switch. _transcribe_with_provider sees a non-empty model name and forwards the invalid Whisper identifier to _run_qwen2_audio, which makes transformers.AutoProcessor.from_pretrained("large-v3") raise and the endpoint returns HTTP 500. The environment variable QWEN2_AUDIO_MODEL is never applied unless the caller overrides the model manually. Defaulting model_name based on the chosen provider (Qwen vs Whisper) would avoid the failure and honour the documented behaviour that Qwen2 Audio is supported out of the box.

Useful? React with 👍 / 👎.

[POWERFULMOVES]

@codex fix comments

[chatgpt-codex-connector]

Summary

• Ensured the /transcribe endpoint defaults to QWEN2_AUDIO_MODEL whenever provider="qwen2-audio" is selected without an explicit whisper_model, preventing invalid Whisper model IDs from being forwarded. pmoves/services/ffmpeg-whisper/server.pyL398-L403

Testing

• ⚠️ pytest pmoves/tests/test_ffmpeg_whisper.py (fails: ModuleNotFoundError: No module named 'fastapi')

View task →