Enable stateful decoding of RNNT over multiple transcribe calls

[titu1994]

Changelog

• Enable transcribe method to store hypothesis and reuse them for stateful multi step predictions
• RNNT transcribe now takes partial_hypotheses as optional input, provided in returned hypothesis from previous transcribe call
• Add abstract batch_concat_states and batch_copy_states support for RNNT models to abstract away state management.
• Force all greedy and beam inference to pack results into Hypothesis objects. They now contain by default the
  • y_sequence : tensor form of int ids of tokens
  • dec_state : CPU tuple of decoder states
  • score: logprob score of current y_sequence
  • (optional) alignments: 2d dangling matrix of token alignments

Example

model = EncDecRNNTModelBPE.restore_from(...)

decoding= model.cfg.decoding
decoding.strategy = 'greedy'
model.change_decoding_strategy(decoding)

# First transcription, stateless
part_hyp, _ = model.transcribe([path to audio files], batch_size=4, return_hypotheses=True)

# Second transcription, restored state
hyp, _ = model.transcribe([path to continuation of audio files], return_hypotheses=True, partial_hypothesis=part_hyp)

[lgtm-com]

This pull request introduces 3 alerts when merging d7598fc into 1f36f32 - view on LGTM.com

new alerts:

• 2 for Unused local variable
• 1 for Unused import

[lgtm-com]

This pull request introduces 3 alerts when merging 6318e3f into 1f36f32 - view on LGTM.com

new alerts:

• 2 for Unused local variable
• 1 for Unused import

[lgtm-com]

This pull request introduces 3 alerts when merging 4b235eb into 9405273 - view on LGTM.com

new alerts:

• 2 for Unused local variable
• 1 for Unused import

[jbalam-nv]

LGTM