Skip silence around hallucinations

[trungkienbkhn]

Same with openai/whisper#1838

[makaveli10]

@trungkienbkhn looks like this slows down the inference. Have you noticed the increase in latency?

[Purfview]

looks like this slows down the inference

Original author mentioned:

...since this also requires extra processing time, we only do this when a probable hallucination is detected.

[trungkienbkhn]

@makaveli10 , hello, sorry for the late reply.
I tested an mp3 audio file (192 seconds) that had a lot of noise with the tiny model and device cuda.
If use hallucination_silence_threshold=2, the avearage execution time total is 5.38s.
And if use the original code and don't use this feature, it's 4.87s.
My code:

model = WhisperModel('tiny', device='cuda')
segments, info = model.transcribe(audio_path, word_timestamps=True, hallucination_silence_threshold=2)

=> Latency has increased a bit. But I found that the transcription quality also improved. So I think it's a trade off, it's not too impactful and is acceptable.