fix(workers): preserve Whisper GPU allocation from Ray resources

[Ahmath-Gadji]

The bug

WhisperPool silently runs Whisper transcription on CPU even when the node has a GPU, making audio/video indexing far slower than it should be.

Root cause

Same failure mode as the Marker GPU regression (#451 / #452), now in the Whisper path.

WhisperPool is created with num_gpus=0 (it's just a dispatcher — see bootstrap.py). Ray therefore clears CUDA_VISIBLE_DEVICES inside the pool actor's process, so torch.cuda.is_available() returns False there even on a GPU node.

_whisper_num_gpus() asked that question from inside the pool actor before deciding how many GPUs to request for the real WhisperActor workers:

return config.loader.local_whisper.whisper_num_gpus if torch.cuda.is_available() else 0

Because CUDA is hidden in the pool process, this returned 0, so the WhisperActors were scheduled with num_gpus=0 and loaded the model on CPU (device = "cuda" if torch.cuda.is_available() else "cpu" → cpu). Ray never reserved a GPU. The API still worked, so it looked like a performance issue rather than a misconfiguration.

The wrong question was "can this pool actor see CUDA?" — it should be "does the Ray cluster have GPU capacity Whisper should request?"

The fix

_whisper_num_gpus() now decides from Ray cluster resources, mirroring _marker_num_gpus():

• configured request <= 0 → stays on CPU (explicit opt-out preserved)
• Ray reports GPU capacity → keep the configured GPU request (e.g. 0.25)
• Ray lookup fails → log a warning and fall back to the old torch.cuda.is_available() check

Adds a regression test covering the hidden-CUDA case plus the config-disabled and Ray-lookup-failure fallbacks.

Summary by CodeRabbit

Release Notes

• Bug Fixes

  • Improved GPU resource detection to increase reliability of GPU allocation.

• Tests

  • Added comprehensive test coverage for GPU selection behavior across different system configurations.

[coderabbitai]

📝 Walkthrough

Walkthrough

_whisper_num_gpus is rewritten to query ray.cluster_resources().get("GPU") for GPU availability instead of using only torch.cuda.is_available(), with a fallback to the CUDA check when the Ray call raises an exception. A new unit test module adds three tests covering the Ray-first, disabled-by-config, and Ray-error-fallback paths.

Changes

Whisper GPU detection via Ray cluster resources

Layer / File(s)	Summary
_whisper_num_gpus Ray-based GPU detection openrag/services/workers/parsers/whisper_workers.py	Returns 0 for non-positive config values; queries ray.cluster_resources().get("GPU") and returns the configured fraction when Ray reports GPUs; falls back to torch.cuda.is_available() if the Ray call raises an exception. Adds an expanded docstring.
Unit tests for _whisper_num_gpus tests/unit/services/workers/parsers/test_whisper_workers.py	Adds a _config helper and three tests: Ray-reported GPUs are used when CUDA is hidden, a non-positive config value returns 0, and a Ray exception triggers the CUDA fallback for both GPU-present and GPU-absent cases.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐇 Hippity-hop, I asked Ray the cluster,
"Are there GPUs? Show me their lustre!"
When Ray threw a fit,
CUDA stepped in a bit—
No GPU left behind, with nary a bluster! 🌟

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 71.43% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely summarizes the main change: fixing GPU allocation for Whisper by using Ray cluster resources instead of relying only on CUDA availability.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/whisper-gpu-allocation

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Ahmath-Gadji]

@CodeRabbit review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.