Fix Silence Sampling Algorithm for ASR Multi-speaker Data Simulator

[stevehuang52]

Signed-off-by: stevehuang52 [email protected]

What does this PR do ?

Replace the previous silence insertion method with a new one that guarantees close approximation to specified mean_silence

Collection: [ASR]

Adding Silence in ASR Data Simulator

Requirements:

1. Sentence durations in each session follow a negative-binomial (NB) distribution
2. Silence ratios in all sessions follows a Beta distribution (range in [0.1])
3. Per silence length should look like a long tail distribution, thus Gamma distribution is used.
4. Silence Uniformity: each speech sentence (overlaps combined) should be followed by some minimum silence
5. Silence Variablity: per-silence durations and sentence durations should be approximately independent (i.e., p-value close to 0)

Parameters:

• NUM_SESSIONS: number of sessions
• MAX_SESS_DUR: maximum session duration
• SAMPLING_RATE: sampling rate for audio
• [NB_COUNT, NB_PROB]: parameters for per sentence duration distribution
• SILENCE_RATIO_MEAN: mean for target silence ratio in all sessions, in (0,1)
• SILENCE_RATIO_VAR: std for target silence ratio in all sessions, set small values (e.g., 0.1) for better approximation to mean, set larger (e.g., 2.0) for more diversity in silence.
• PER_SILENCE_VAR: std for individual silence length, default to 20 for achieving p-value=0.1 to de-correlate speech and silence lengths
• [PER_SILENCE_MIN,PER_SILENCE_MAX]: mix and max of per silence duration in seconds, max=-1 for no constraint

Algorithm:

MAX_SESSION_LEN = MAX_SESS_DUR * SAMPLING_RATE
MIN_SILENCE_LEN = SILENCE_RATIO_MIN * SAMPLING_RATE
MAX_SILENCE_LEN = min(SILENCE_RATIO_MAX * SAMPLING_RATE, MAX_SESSION_LEN)

sessions = []
for i in range(NUM_SESSIONS):
    curr_session = []

    curr_sess_len = 0
    curr_speech_len = 0
    curr_silence_len = 0

    a = SILENCE_RATIO_MEAN ** 2 * (1 - SILENCE_RATIO_MEAN) / SILENCE_RATIO_VAR - SILENCE_RATIO_MEAN
    b = SILENCE_RATIO_MEAN * (1 - SILENCE_RATIO_MEAN) ** 2 / SILENCE_RATIO_VAR - (1 - SILENCE_RATIO_MEAN)
    sess_silence_mean = Beta(a, b).rvs()

    while curr_sess_len < MAX_SESSION_LEN:
        speech_len = NB(NB_COUNT, NB_PROB).rvs()
        sentence = build_sentence(speech_len,curr_sess_len,  MAX_SESSION_LEN)
        
        curr_session += sentence
        curr_sess_len += len(sentence)
        curr_speech_len += len(sentence)

        if curr_sess_len >= MAX_SESSION_LEN:
            break
        
        # dynamically adjust silence mean to achieve the overall mean
        silence_mean = max(1, MIN_SIL1ENCE_LEN, (sess_silence_mean * curr_sess_len - curr_silence_len) / (1 - sess_silence_mean))

        # sampling with large std to de-correlate with previous sentence length
        silence_len = Gamma(a=silence_mean ** 2 / PER_SILENCE_VAR, scale=PER_SILENCE_VAR / silence_mean).rvs()  

        # enforce valid length
        silence_len = min(max(MIN_SILENCE_LEN, silence_len), MAX_SILENCE_LEN, max_session_len - curr_sess_len)  
        
        silence = add_silence(silence_len)
        
        curr_session += silence
        curr_sess_len += silence_len
        curr_silence_len += silence_len

    sessions.append(curr_session)

Notes

• Harder to get desired silence ratio with shorter session length
  • E.g., sess_len=20s, mean_silence=0.35, 500 hours -> actual ratio 0.24
  • E.g., sess_len=120s. mean_silence=0.3, 100 hours -> actual ratio ~0.3
• Silence lengths distribution is approximately exponential, and reducing silence ratio can reduce the avg silence length
  • E.g., mean_silence=0.1, std=0.05 -> mean per-silence length 0.2s
  • E.g., mean_silence=0.2, std=0.5 -> mean per-silence length 2.4s

[tango4j]

Notebook is tested, works with no problem. the new script was missing a license template, so I added. I will approve as soon as it passes the test.

[tango4j]

Notebooks and data simulation code both tested. Very nice work by stevehuang, thanks.