Adjust stochastic depth dropout probability calculation

[anteju]

What does this PR do?

This PR adjusts calculation of stochastic depth dropout probability.
Currently, calculation fails for 1-layer ConformerEncoder with the default setup.

The culprit is calculation of the probability in compute_stochastic_depth_drop_probs for linear mode:

layer_drop_probs += [l / (L - 1) * stochastic_depth_drop_prob for l in range(L)]

since in this case L-1 == 0.

Proposed solution

Change calculation for the linear mode to the following

# we start with 1/L * drop_prob and and end with the desired drop probability.
layer_drop_probs += [l / L * stochastic_depth_drop_prob for l in range(1, L + 1)]

This can handle setup with a single layer and is consistent with the probability calculation for the uniform mode.

Example

An example of calculated probabilities:

num_layers = 7
stochastic_depth_start_layer = 3
stochastic_depth_drop_prob = 0.5

layer	0	1	2	3	4	5	6
uniform	0.0	0.0	0.0	0.5	0.5	0.5	0.5
linear original	0.0	0.0	0.0	0.0	0.167	0.333	0.5
linear proposed	0.0	0.0	0.0	0.125	0.25	0.375	0.5

Collection: ASR

Changelog

• Add specific line by line info of high level changes in this PR.

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you add or update any necessary documentation?
• Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

• New Feature
• Bugfix
• Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

• Related to # (issue)

[Kipok]

Thanks Ante, looks good!