[W2V2 with LM] Fix decoder test with params

[sanchit-gandhi]

What does this PR do?

Fixes #21226. This test started failing due to a PyPI update of pyctcdecode, which incorporated a number of bug fixes to the LM decode method (see kensho-technologies/pyctcdecode#107 (comment)).

This PR modifies the decoding params, such that the same outputs are obtained for pyctcdecode v0.4.0 and v0.5.0 (latest) - hence the test should pass irrespective of the package version while still verifying correctness of the outputs. The PR also adds tests for the LM and logit scores.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[ydshieh]

Thank you @sanchit-gandhi !

(BTW - off topic - are names decoded_decoder or decoded_processor in this file intended to be so? This sounds like names for decoder or processor objects, and I find them a bit confusing.)

[sgugger]

Thanks for the fix!

[sanchit-gandhi]

Indeed the var names are a bit confusing @ydshieh! They follow the same convention throughout the test file, where decoded_processor refers to the LM outputs decoded by the HF processor class, and decoded_decoder refers to the LM outputs decoded by the pyctc decoder class.

I'll update these var names in a follow-up PR to make them a bit more intuitive 👍