Print best and worst results in a WER report.

[DanBmh]

This is also correcting the error that the results with the highest WER instead of the lowest WER are printed.

[community-tc-integration]

No Taskcluster jobs started for this pull request

The `allowPullRequests` configuration for this repository (in `.taskcluster.yml` on the
default branch) does not allow starting tasks for this pull request.

[lissyx]

@DanBmh Can I ask you why you think it's useful to print the best ones ?

[DanBmh]

I didnt know for some time that the examples are the worst predictions from the test set. I thought they were chosen randomly and i was wondering why they were always so bad. So i did ignore the examples after a while. After i found the flag description (which by the way says they are the best results as lower is better) i understood they are the worst results, This is makes more sense than just printing random ones too.

So i think printing both the best and the worst will provide new users with an intuitive way to see that we output the worst results.
Another benefit is that if you have a bad performing network you can still see the network learnt something:)

Maybe the best idea is to print median results too, so that you get a more realistic estimate of the prediction quality? Like this:
1
2
[...]
5
6
[...]
9
10

[victornoriega]

I think that when you're modeling and trying a lot of configurations, you want to know the flaws of your model but also the best it can show in which cases it can excel. I also think this should be an additional flag to DeepSpeech.py, not something by default.

[lissyx]

Maybe the best idea is to print median results too, so that you get a more realistic estimate of the prediction quality? Like this:
1
2
[...]
5
6
[...]
9
10

It makes sense, but I have to admit this is not something that ever crossed our mind.

[lissyx]

@DanBmh You might need to factorize / apply that to evaluate.py as well as evaluate_tflite.py.

[DanBmh]

evaluate_tflite.py does not print any samples. Shall i add it?

[reuben]

Thanks for the PR!

[lissyx]

evaluate_tflite.py does not print any samples. Shall i add it?

I'd say no then.

[lissyx]

LGTM, thanks!

[lissyx]

I've triggered some PR to have TaskCluster running on that.

[lissyx]

Thanks @DanBmh !

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.