-
Notifications
You must be signed in to change notification settings - Fork 5.4k
WIP: TDNN+LSTM and TDNN scripts #1212
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Good results even in AMI. #old tdnn baseline
%WER 40.6 | 14891 94509 | 63.0 19.8 17.2 3.7 40.6 65.1 | 0.594 | exp/sdm1/chain_cleaned/tdnn_sp_bi_ihmali/decode_dev/ascore_9/dev_hires_o4.ctm.filt.sys
%WER 44.8 | 14251 89975 | 58.7 21.1 20.2 3.5 44.8 64.3 | 0.558 | exp/sdm1/chain_cleaned/tdnn_sp_bi_ihmali/decode_eval/ascore_9/eval_hires_o4.ctm.filt.sys
# new tdnn baseline (i.e., w/o pretraining and using swbd/tdnn_7h architecture)
%WER 39.9 | 14249 94516 | 63.8 19.6 16.6 3.6 39.9 68.1 | 0.589 | exp/sdm1/chain_cleaned/tdnn1b_sp_bi_ihmali/decode_dev/ascore_9/dev_hires_o4.ctm.filt.sys
%WER 43.9 | 13109 89968 | 59.5 21.2 19.3 3.4 43.9 69.8 | 0.557 | exp/sdm1/chain_cleaned/tdnn1b_sp_bi_ihmali/decode_eval/ascore_9/eval_hires_o4.ctm.filt.sys
# tdnn+lstm (based on swbd/tdnn_lstm_1b)
%WER 38.9 | 14798 94506 | 65.0 19.5 15.5 3.9 38.9 64.9 | 0.636 | exp/sdm1/chain_cleaned/tdnn_lstm1a_sp_bi_ihmali_ld5/decode_dev/ascore_9/dev_hires_o4.ctm.filt.sys
%WER 42.2 | 13507 89988 | 61.3 21.3 17.4 3.5 42.2 67.7 | 0.602 | exp/sdm1/chain_cleaned/tdnn_lstm1a_sp_bi_ihmali_ld5/decode_eval/ascore_9/eval_hires_o4.ctm.filt.sys |
|
Wow, great!
…On Thu, Nov 24, 2016 at 12:54 AM, Vijayaditya Peddinti < ***@***.***> wrote:
Good results even in AMI.
#old tdnn baseline
%WER 40.6 | 14891 94509 | 63.0 19.8 17.2 3.7 40.6 65.1 | 0.594 | exp/sdm1/chain_cleaned/tdnn_sp_bi_ihmali/decode_dev/ascore_9/dev_hires_o4.ctm.filt.sys
%WER 44.8 | 14251 89975 | 58.7 21.1 20.2 3.5 44.8 64.3 | 0.558 | exp/sdm1/chain_cleaned/tdnn_sp_bi_ihmali/decode_eval/ascore_9/eval_hires_o4.ctm.filt.sys
# new tdnn baseline (i.e., w/o pretraining and using swbd/tdnn_7h architecture)
%WER 39.9 | 14249 94516 | 63.8 19.6 16.6 3.6 39.9 68.1 | 0.589 | exp/sdm1/chain_cleaned/tdnn1b_sp_bi_ihmali/decode_dev/ascore_9/dev_hires_o4.ctm.filt.sys
%WER 43.9 | 13109 89968 | 59.5 21.2 19.3 3.4 43.9 69.8 | 0.557 | exp/sdm1/chain_cleaned/tdnn1b_sp_bi_ihmali/decode_eval/ascore_9/eval_hires_o4.ctm.filt.sys
# tdnn+lstm (based on swbd/tdnn_lstm_1b)
%WER 38.9 | 14798 94506 | 65.0 19.5 15.5 3.9 38.9 64.9 | 0.636 | exp/sdm1/chain_cleaned/tdnn_lstm1a_sp_bi_ihmali_ld5/decode_dev/ascore_9/dev_hires_o4.ctm.filt.sys
%WER 42.2 | 13507 89988 | 61.3 21.3 17.4 3.5 42.2 67.7 | 0.602 | exp/sdm1/chain_cleaned/tdnn_lstm1a_sp_bi_ihmali_ld5/decode_eval/ascore_9/eval_hires_o4.ctm.filt.sys
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#1212 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ADJVu6C0c5GbxRpmyqAV0Kp5nboQJk3Eks5rBSaRgaJpZM4K7Bwi>
.
|
| #!/bin/bash | ||
|
|
||
| # same as 1a, but with more TDNN layers between each LSTM | ||
| #System 1a 1c |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
adding more TDNNs in between lstms improves results while adding TDNNs after LSTMs made it worse.
|
Interesting. BTW, merge this whenever you see fit.
…On Fri, Nov 25, 2016 at 1:15 PM, Vijayaditya Peddinti < ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In egs/ami/s5b/local/chain/tuning/run_tdnn_lstm_1c.sh
<#1212 (review)>:
> @@ -0,0 +1,292 @@
+#!/bin/bash
+
+# same as 1a, but with more TDNN layers between each LSTM
+#System 1a 1c
adding more TDNNs in between lstms improves results while adding TDNNs
after LSTMs made it worse.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1212 (review)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ADJVu1SPlDCJS4bZu9En0O3g5E8nwPmxks5rByXbgaJpZM4K7Bwi>
.
|
|
Results on IHM. Gains are consistent. TDNN+LSTM
%WER 21.1 | 13098 94474 | 81.7 10.0 8.3 2.8 21.1 54.0 | -0.033 | exp/ihm/chain_cleaned/tdnn_lstm1g_sp_bi_ld5/decode_dev/ascore_11/dev_hires.ctm.filt.sys
%WER 20.9 | 12643 89977 | 81.6 11.3 7.1 2.5 20.9 52.0 | 0.057 | exp/ihm/chain_cleaned/tdnn_lstm1g_sp_bi_ld5/decode_eval/ascore_11/eval_hires.ctm.filt.sys
TDNN
%WER 22.4 | 13098 94487 | 80.6 10.9 8.5 3.0 22.4 55.4 | 0.084 | exp/ihm/chain_cleaned/tdnn_sp_bi/decode_dev/ascore_10/dev_hires.ctm.filt.sys
%WER 22.6 | 12643 89973 | 80.2 12.7 7.1 2.7 22.6 53.1 | 0.158 | exp/ihm/chain_cleaned/tdnn_sp_bi/decode_eval/ascore_10/eval_hires.ctm.filt.sys
|
# 1i is same as 1g but with TDNN output dim 1024 instead of 512
# (num-params 1g:21309812 1i: 43447156)
System 1g 1i
WER on dev 38.3 37.6
WER on eval 41.6 40.9
Final train prob -0.138017 -0.114135
Final valid prob -0.238659 -0.245208
Final train prob (xent) -1.66834 -1.47648
Final valid prob (xent) -2.17419 -2.16365 |
|
nice! |
|
Overall tuning of tdnn+lstm recipe gave a 2% abs. reduction in WER.
System 1a 1i
WER on dev 38.9 37.6
WER on eval 42.2 40.9
Final train prob -0.142585 -0.114135
Final valid prob -0.251197 -0.245208
Final train prob (xent) -1.73176 -1.47648
Final valid prob (xent) -2.26965 -2.16365Edited : Previous comparison was with 1b, but it should have been with 1a. |
|
This result looks good. Will it be helpful for fisher_swbd? I can have
it running when I have computing resources...
Xingyu
在 2016/11/29 4:01, Daniel Povey 写道:
…
nice!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#1212 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ADKpxHwi4vUwHz_WQeSeMN-ELl5K3Gzoks5rCzMBgaJpZM4K7Bwi>.
|
|
We expect it to help there too. You can look at swbd/tdnn_lstm_1b.sh recipe
or wait for 1d which is being tested now and expected to be better (based
on AMI results).
…--Vijay
On Tue, Nov 29, 2016 at 2:01 AM, Xingyu Na ***@***.***> wrote:
This result looks good. Will it be helpful for fisher_swbd? I can have
it running when I have computing resources...
Xingyu
在 2016/11/29 4:01, Daniel Povey 写道:
>
> nice!
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <#1212 (comment)>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-
auth/ADKpxHwi4vUwHz_WQeSeMN-ELl5K3Gzoks5rCzMBgaJpZM4K7Bwi>.
>
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#1212 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ADtwoO76yPCzaFncEVSX7R63g27YZqDYks5rC83DgaJpZM4K7Bwi>
.
|
|
You might want to wait a few days until we check in the 'fast-lstm' code
and scripts. Then your experiments would finish a lot faster.
On Tue, Nov 29, 2016 at 2:08 AM, Vijayaditya Peddinti <
notifications@github.com> wrote:
… We expect it to help there too. You can look at swbd/tdnn_lstm_1b.sh recipe
or wait for 1d which is being tested now and expected to be better (based
on AMI results).
--Vijay
On Tue, Nov 29, 2016 at 2:01 AM, Xingyu Na ***@***.***>
wrote:
> This result looks good. Will it be helpful for fisher_swbd? I can have
> it running when I have computing resources...
>
> Xingyu
>
>
> 在 2016/11/29 4:01, Daniel Povey 写道:
> >
> > nice!
> >
> > —
> > You are receiving this because you are subscribed to this thread.
> > Reply to this email directly, view it on GitHub
> > <#1212 (comment)>,
> > or mute the thread
> > <https://github.com/notifications/unsubscribe-
> auth/ADKpxHwi4vUwHz_WQeSeMN-ELl5K3Gzoks5rCzMBgaJpZM4K7Bwi>.
>
> >
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <#1212 (comment)>,
or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/
ADtwoO76yPCzaFncEVSX7R63g27YZqDYks5rC83DgaJpZM4K7Bwi>
> .
>
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1212 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ADJVuxZXOOgQQOiuxNaYxWZh2usuaI_-ks5rC8-GgaJpZM4K7Bwi>
.
|
|
@danpovey Is it possible to move Vimal's code or your code as a branch to the main Kaldi repo, so that we can all push our changes to that branch rather than branching from each other's PRs ? |
|
OK, I'll push to 'fast_lstm' branch in main repo; includes Vimal's changes.
…On Tue, Nov 29, 2016 at 4:57 PM, Vijayaditya Peddinti < ***@***.***> wrote:
@danpovey <https://github.com/danpovey> Is it possible to move Vimal's
code or your code as a branch to the main Kaldi repo, so that we can all
push our changes to that branch rather than branching from each other's PRs
?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1212 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ADJVuxNFgq0onH0nc-6nLMnROUHZJ2bBks5rDJ-vgaJpZM4K7Bwi>
.
|
d4b03e0 to
44a848b
Compare
swbd : added results for the TDNN recipe which uses the subset-dim option
44a848b to
426e0d6
Compare
This PR has been created to keep people informed about the latest TDNN+LSTM and TDNN experiments.
ami : added xconfig TDNN and TDNN+LSTM scripts for AMI.
Results soon.
swbd : added results for the TDNN recipe which uses the subset-dim option (worse than normal TDNN)