add hlg decode

[aluminumbox]

We add hlg decode using k2. Now we can use k2 to compile a HLG, and decode using python with cuda.
We provide two decoding algorithm. One is onebest, which is same as ctc_prefix_beam_search with lm score, another is attention rescore.
Notice that in attention rescore, we have 3 different scale parameter, which are lm_scale, decoder_scale and r_decoder_scale.
Special thanks to K2 groups great work!

[robin1001]

move it into hlg_onebest & hlg_rescore, so we can run it without k2 & icefall.

[aluminumbox]

To avoid duplicate code, we use move these import into try except

[robin1001]

How about HLG(k2 LM), so it is easy to understand for the users.

[aluminumbox]

Done

[robin1001]

please see inline.

[csukuangfj]

What is the motivation to add HLG from k2?

[aluminumbox]

What is the motivation to add HLG from k2?

k2 can do batch decoding with cuda, and it has python interface.
It is easier to use and faster compared with wenet runtime tlg.
Also, it can extract lm score from tot score, so we have more hyperparameter to tune when decoding. And the results on aishell is 4.32, which is better than runtime tlg lm, 4.40

[csukuangfj]

What is the motivation to add HLG from k2?

k2 can do batch decoding with cuda, and it has python interface. It is easier to use and faster compared with wenet runtime tlg

Thanks! Do you have some benchmarks to share?

[csukuangfj]

Do we need to add

exit 1

to stop processing?

[aluminumbox]

Done

[csukuangfj]

Shall we skip this step if the file is already generated?

[aluminumbox]

Done

[aluminumbox]

What is the motivation to add HLG from k2?

k2 can do batch decoding with cuda, and it has python interface. It is easier to use and faster compared with wenet runtime tlg

Thanks! Do you have some benchmarks to share?

The wer performance is already compared in README. For speed comparision, wenet runtime default using batchsize=1, which is not comparable with hlg decode using batchsize=16. I will do the speed benchmark later in a fair scenario

[robin1001]

It is more convienent to use FSA in python than c++ in openfst, especially for the beginners. And Dan thinks it is okay to support k2 in wenet.

[robin1001]

Besides, we can use k2 with GPU.

[zw76859420]

We add hlg decode using k2. Now we can use k2 to compile a HLG, and decode using python with cuda. We provide two decoding algorithm. One is onebest, which is same as ctc_prefix_beam_search with lm score, another is attention rescore. Notice that in attention rescore, we have 3 different scale parameter, which are lm_scale, decoder_scale and r_decoder_scale. Special thanks to K2 groups great work!

Thank you for sharing, but we would like to know how we can access HLG Decoding using c++？

[zw76859420]

It‘s unpossible that we load the dynamic libraries provided by k2 and wenet at the same time.
So, can we combine both wenet and k2 in a simple way?

[csukuangfj]

It‘s unpossible that we load the dynamic libraries provided by k2 and wenet at the same time.

Why is it impossible?

Have you used the same version of PyTorch and CUDA to compile k2 and wenet?