CPU can incur a slow path on non-contiguous magnitudes

[vbayanag]

What does this PR do?

stft[..., :-1] produces a non-contiguous view. Asobserved on MI355X ROCm vLLM latest build image, the audio preprocessing function mel_filters.T @ magnitudes falls onto a strided-GEMM path that runs ~8x slower (~43 ms vs ~3 ms for a 30 s audio window), dominating Whisper feature extraction.

Forcing magnitudes contiguous costs ~0.18 ms but restores the fast matmul path, cutting per-request audio preprocessing substantially.

Fixes # (issue)

Make the CPU op run on contiguous memory layout for ~8x speedup

Before/after (single 30 s window, ROCm vLLM image)

stage	non-contiguous	contiguous
mel_filters.T @ magnitudes	~43 ms	~3 ms
.contiguous() overhead	–	~0.18 ms

Benchmark Code

import torch
import time 
import numpy as np
from transformers import WhisperFeatureExtractor

torch.manual_seed(0)
feature_extractor = WhisperFeatureExtractor()
n_fft = feature_extractor.n_fft
hop_length = feature_extractor.hop_length
# Mirror `_torch_extract_fbank_features`: the window is built from n_fft.
window = torch.hann_window(n_fft)
# One full Whisper window of audio (chunk_length * sampling_rate samples).
waveform = torch.randn(feature_extractor.n_samples, dtype=torch.float32)

stft = torch.stft(waveform, n_fft, hop_length, window=window, return_complex=True)
magnitudes_non_contiguous = stft[..., :-1].abs() ** 2
magnitudes_contiguous = magnitudes_non_contiguous.contiguous()

mel_filters = torch.from_numpy(feature_extractor.mel_filters).to(torch.float32)

def _median_ms(magnitudes, iters=20, warmup=5):
    for _ in range(warmup):
        mel_filters.T @ magnitudes
    timings = []
    for _ in range(iters):
        start = time.perf_counter()
        mel_filters.T @ magnitudes
        timings.append((time.perf_counter() - start) * 1e3)
    timings.sort()
    return np.mean(timings) #timings[len(timings) // 2]

with torch.inference_mode():
    non_contiguous_ms = _median_ms(magnitudes_non_contiguous)
    contiguous_ms = _median_ms(magnitudes_contiguous)

print(
    f"\n[whisper mel matmul] non-contiguous={non_contiguous_ms:.3f} ms "
    f"contiguous={contiguous_ms:.3f} ms "
    f"speedup={non_contiguous_ms / contiguous_ms:.2f}x"
)

Code Agent Policy

The Transformers repo is currently being overwhelmed by a large number of PRs and issue comments written by
code agents. These often are low-quality, or fix extremely minor issues that occur rarely or never in practice.
As a result, we're instituting a rule that first-time contributors should not use code agents to submit PRs or issues.
We'd also ask autonomous "OpenClaw"-like agents not to open any PRs or issues.

Issues/PRs from first-time contributors that violate this rule will probably just be closed without review, and we
might block you, especially if you open more than one or appear to be deliberately ignoring this. We especially do not
want new contributors to jump in on random issues to contribute an agent-written fix. This creates lots of noise
for reviewers and other users and will almost certainly get you blocked.

For more information, please read CONTRIBUTING.md.

• (First-time contributors only): I confirm that this PR description and code is not written by an LLM or code agent

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline and the
  Pull Request checks?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes according to the guidelines?
• Did you write any new necessary tests?

Who can review?

@eustlb @ebezzam @vasqu

[vasqu]

Happy to add the fix on 2 conditions

1. Shorten the comment --> cpu can incurr a slow path on non-contiguous magnitutes
2. Add a fast test that can show this or a benchmark with this PR so we can verify as well

[vbayanag]

Happy to add the fix on 2 conditions

1. Shorten the comment --> cpu can incurr a slow path on non-contiguous magnitutes
2. Add a fast test that can show this or a benchmark with this PR so we can verify as well

Hi @vasqu, the posted benchmarks in the description are from before/after change. Kindly let me know if you need anymore information.

[github-actions]

Thank you for your contribution 🤗!

CI Security Gate — automatic approval blocked

This PR was not automatically approved for CI because the security gate failed.

Possible reasons:

• The PR touches 50 or more files — only PRs with fewer than 50 changed files are automatically approved
• A changed file is outside the allowed directories (src/, tests/, docs/, utils/), has a disallowed extension (only .py, .txt, .md permitted outside tests/ and docs/), or is not .md/.yml inside docs/
• A new high-severity security issue was detected in the changed Python files (Bandit check)

See the workflow run for the exact violations.

A maintainer can review and manually approve CI if a finding is a false positive.

[vasqu]

@vbayanag I meant like a small benchmark script :D the numbers are impressive but imo for anyone stumbling on this, it's always nice to have the exact code that produced the numbers along with it

[vbayanag]

@vbayanag I meant like a small benchmark script :D the numbers are impressive but imo for anyone stumbling on this, it's always nice to have the exact code that produced the numbers along with it

Hi @vasqu, I added a test case in the relevant test location. Please check.

Local testing shows 5x uplift. This number is different from what's in description, likely due to difference in the way metrics were extracted. The initial posted numbers were extracted from feature_extractor API call from the trace file of end-to-end inference testing of whisper model, whereas the uplift observed now is localised only to the matmul operation.

[whisper mel matmul] non-contiguous=10.269 ms contiguous=1.999 ms speedup=5.14x

System details:

1. Docker pull and launch vllm/vllm-openai-rocm:latest image

docker run -it  --group-add=video    --ipc=host    --cap-add=SYS_PTRACE    --security-opt seccomp=unconfined    --device /dev/kfd    --device /dev/dri -v /:/data --entrypoint ''  vllm/vllm-openai-rocm:latest /bin/bash

2. Install depenencies (optional, only required for other test cases to run smoothly)

apt-get update && apt-get install -y ffmpeg
pip install soundfile librosa torchcodec==0.12

3. Run the python unit test inside your clone dir

PYTHONPATH=src python3 -m pytest     tests/models/whisper/test_feature_extraction_whisper.py -k contiguous_magnitudes -sv

[vasqu]

Ok sorry for backtracking a bit but I'd rather have the benchmark within the PR description. A benchmark as test is probably brittle, wdyt?

My goal is just to have documentation that shows why it was done and for ppl to repro

[vbayanag]

Hi @vasqu, I think that this small test case would be evidence of the change I am submitting. By having a test case that's part of the code, it stays persistent and can be removed upon future rebuttals.

If you still think benchmark is unnecessary, I can remove that part from the test case and leave the rest alone. I can post the benchmark script on description. Kindly let me know what you think

[vasqu]

I can post the benchmark script on description.

Yea I think this is the better option tbh. It doesn't really make much sense as standalone test. We should refer to the benchmark script in some shape or form, e.g. a PR reference (that has it in the description) or a gh gist.

[vbayanag]

Done.

1. I removed benchmark code from test case and added in the description.
2. The test case now only verifies the validity of the matrix when changed to contiguous.

[vasqu]

Thanks, one small nit and can you run make style for CI?

[vasqu]

Let's also add a reference to this PR in the description please

[vbayanag]

Thanks, one small nit and can you run make style for CI?

How to do that?

[vasqu]

You just have to run make style on the root of the folder

[vbayanag]

You just have to run make style on the root of the folder

Done

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: whisper

[github-actions]

---

CI recap

Dashboard: View test results in Grafana
Latest run: 29865079473:2
Result: success | Jobs: 6 | Tests: 981 | Failures: 0 | Duration: 5m 14s

[vasqu]

Merging it, thanks for your patience!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.