[andyluo7]

Exact CUDA replay result - September 30, 2026, 23:58 UTC

CUDA #16454 at66b6169ee finishes3/3jobs,exit0nonsoft. Its diagnostic calls completed; this is not a fullDocumentationCIpass. Results,WAVandoriginaltranscription areSHA256verified. Samefixedc30image,sourcehelpers,WAV,Whisper-smallcheckpoint,H10080GB/cuda1,Torch2.13.0+cu130,Whisper20250625.

Decoder	Two similarities	Existing0.8gate
Originalgreedy	0.0799607318 / 0.0799607318	Fail / Fail
FP32greedy	0.0799607318 / 0.0799607318	Fail / Fail
Upstreamtemperaturefallback(0,.2,.4,.6,.8,1)	0.9870362803 / 0.9870362803	Pass / Pass

Seed0beforeeachcall. Thefallbackkeeps10segmentsattemperature0anddecodestherepetitivefinalsegmentat0.4;compressionratiofalls8.08475to0.92982. OriginalandFP32retainthe427repeatedfcharacters. Beam5from#16450alsofailed. Fallbackretainsashortclippedending,soitdoesnotproveperfectgeneration;itsupportedlyremovestheASRrepetitionartifactwithoutloweringthequalitygate.

SeparateDocumentation-onlyopt-inhelperrepairisbeingprepared;productiongenerationandotherASRcallerdefaultsunmodified. KeepthisvalidationdraftandtemporaryCUDAentrypipelineunmerged.

Current diagnostic head

Head: 66b6169ee9b532173d0bed748c6bd0d2262ba189. The next replay implements two FP32 controls and two seed-0 temperature-schedule fallbacks alongside original greedy repeats. Syntax, diff and all changed-file hooks pass. GPU execution is pending. Historical results below are explicitly tied to e8104d1.

First replay result - September 30, 2026, 23:31 UTC

CUDA #16450 at e8104d1f4644620260e920213da1423491000de6 completes with exit 0, non-soft; this means the diagnostic calls ran, not a Documentation quality pass. The retained result JSON SHA-256 matches Buildkite metadata: 51b518db3c72acbcd5068ab95abc906ee522983610eddaf215c61027462dea1a.

• Original greedy twice: similarity 0.07996073180416986, exact original transcript, repeated-f run of 427 characters.
• Temperature-zero beam-size-5 twice: the same score and transcript. This candidate does not fix the failure.
• The final segment at 52.56-56.62s has compression ratio 8.0847457627, above Whisper's default 2.4 fallback threshold; avg_logprob approximately -0.1600 and no_speech_prob approximately 0.3248. Scalar temperature zero allows no additional decoding attempt.
• Image/source/input/model hashes verified, Whisper20250625, Torch2.13.0+cu130, CUDA13.0, H10080GB/cuda:1, model-small hash9ecf779972d90ba49c06d968637d720dd632c55bbf19d441fb42bf17a411e794.

The next diagnostic compares FP32 greedy and the upstream temperature fallback schedule with seed 0, twice each, against two original greedy controls. The fixed audio/source/image and unchanged 0.8 quality gate are retained. No global helper setting has changed. This PR must remain draft and its temporary CUDA entry pipeline must not merge.

Purpose

Draft validation experiment for the CUDA Documentation failure in #8344, build 16446, at c30e63d04dc9a54c5c4b43235eb2ce535fe68df3.

The retained modality-control-003 WAV is 56.650625 seconds. Original OpenAI Whisper-small returned a repeated-letter tail and similarity 0.0799607318 against the unchanged 0.8 gate. Independent CPU transcription with another model is coherent, but does not reproduce the original helper.

This experiment downloads the exact WAV and transcript from Buildkite and verifies their SHA-256 hashes. It uses the published c30 CUDA image on two H100s and pins Whisper to logical device 1, as observed in the failed job. Both source helper hashes are checked before runtime. The shared helper and Documentation quality gate remain unchanged.

Experiment

Run original settings twice and deterministic beam size 5 twice, alternating the runs. Keep temperature 0, word timestamps, no previous-text conditioning, and language autodetection. Save complete responses and per-segment metrics, model hash, package versions, device/capability and existing text-similarity score.

A successful diagnostic job means the calls completed. The recorded quality results decide whether a candidate deserves a separate source fix; this PR is not a replacement Documentation pass.

Validation

• Exact input download and helper-hash preflight passed locally.
• Python compilation and git diff checks passed.
• All applicable pre-commit hooks passed, including mypy, lint, SPDX and Buildkite schema.
• Rendered pipeline contains one bounded H100 experiment with retained input/result artifacts.
• No local Torch/vLLM runtime was installed. Actual CUDA replay is pending.

Merge gate

Keep draft; validation only. The temporary CUDA entry pipeline runs only this experiment and must not be merged. A qualified decoder correction will be proposed separately. The passing AMD c30 bundle remains unchanged.

[hsliuustc0106]

This PR touches .buildkite/cuda/, tools/ci/ (3 files). Based on CODEOWNERS coverage of the changed files, the most-related reviewers appear to be:

@yenuo26 @congw729 @NickCao

Could one of you take a look when you get a chance? Thanks!

[andyluo7]

Closing as an obsolete diagnostic/evidence-only PR. Current main AMD merge builds #13240, #13241, and #13242 are effectively green, and this stale stacked branch should not merge as-is. If the underlying issue still reproduces, the relevant source change should be extracted into a focused PR rebased on current main.