Skip to content

Conversation

@vijayaditya
Copy link
Contributor

This PR has been created to keep people informed about the latest TDNN+LSTM and TDNN experiments.
ami : added xconfig TDNN and TDNN+LSTM scripts for AMI.
Results soon.

swbd : added results for the TDNN recipe which uses the subset-dim option (worse than normal TDNN)

+#System                  tdnn_7h   tdnn_7j
+#WER on train_dev(tg)      13.84     14.15
+#WER on train_dev(fg)      12.84     12.96
+#WER on eval2000(tg)        16.5      16.8
+#WER on eval2000(fg)        14.8      15.1
+#Final train prob     -0.0889771-0.0910883
+#Final valid prob      -0.113102 -0.112464
+#Final train prob (xent)       -1.2533  -1.31768
+#Final valid prob (xent)      -1.36743  -1.41603

@vijayaditya
Copy link
Contributor Author

Good results even in AMI.

#old tdnn baseline
%WER 40.6 | 14891 94509 | 63.0 19.8 17.2 3.7 40.6 65.1 | 0.594 | exp/sdm1/chain_cleaned/tdnn_sp_bi_ihmali/decode_dev/ascore_9/dev_hires_o4.ctm.filt.sys
%WER 44.8 | 14251 89975 | 58.7 21.1 20.2 3.5 44.8 64.3 | 0.558 | exp/sdm1/chain_cleaned/tdnn_sp_bi_ihmali/decode_eval/ascore_9/eval_hires_o4.ctm.filt.sys

# new tdnn baseline (i.e., w/o pretraining and using swbd/tdnn_7h architecture)
%WER 39.9 | 14249 94516 | 63.8 19.6 16.6 3.6 39.9 68.1 | 0.589 | exp/sdm1/chain_cleaned/tdnn1b_sp_bi_ihmali/decode_dev/ascore_9/dev_hires_o4.ctm.filt.sys
%WER 43.9 | 13109 89968 | 59.5 21.2 19.3 3.4 43.9 69.8 | 0.557 | exp/sdm1/chain_cleaned/tdnn1b_sp_bi_ihmali/decode_eval/ascore_9/eval_hires_o4.ctm.filt.sys

# tdnn+lstm (based on swbd/tdnn_lstm_1b) 
%WER 38.9 | 14798 94506 | 65.0 19.5 15.5 3.9 38.9 64.9 | 0.636 | exp/sdm1/chain_cleaned/tdnn_lstm1a_sp_bi_ihmali_ld5/decode_dev/ascore_9/dev_hires_o4.ctm.filt.sys
%WER 42.2 | 13507 89988 | 61.3 21.3 17.4 3.5 42.2 67.7 | 0.602 | exp/sdm1/chain_cleaned/tdnn_lstm1a_sp_bi_ihmali_ld5/decode_eval/ascore_9/eval_hires_o4.ctm.filt.sys

@danpovey
Copy link
Contributor

danpovey commented Nov 24, 2016 via email

#!/bin/bash

# same as 1a, but with more TDNN layers between each LSTM
#System 1a 1c
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adding more TDNNs in between lstms improves results while adding TDNNs after LSTMs made it worse.

@danpovey
Copy link
Contributor

danpovey commented Nov 25, 2016 via email

@vijayaditya
Copy link
Contributor Author

Results on IHM. Gains are consistent.

TDNN+LSTM
%WER 21.1 | 13098 94474 | 81.7 10.0 8.3 2.8 21.1 54.0 | -0.033 | exp/ihm/chain_cleaned/tdnn_lstm1g_sp_bi_ld5/decode_dev/ascore_11/dev_hires.ctm.filt.sys
%WER 20.9 | 12643 89977 | 81.6 11.3 7.1 2.5 20.9 52.0 | 0.057 | exp/ihm/chain_cleaned/tdnn_lstm1g_sp_bi_ld5/decode_eval/ascore_11/eval_hires.ctm.filt.sys

TDNN
%WER 22.4 | 13098 94487 | 80.6 10.9 8.5 3.0 22.4 55.4 | 0.084 | exp/ihm/chain_cleaned/tdnn_sp_bi/decode_dev/ascore_10/dev_hires.ctm.filt.sys
%WER 22.6 | 12643 89973 | 80.2 12.7 7.1 2.7 22.6 53.1 | 0.158 | exp/ihm/chain_cleaned/tdnn_sp_bi/decode_eval/ascore_10/eval_hires.ctm.filt.sys

@vijayaditya
Copy link
Contributor Author

# 1i is same as 1g but with TDNN output dim 1024 instead of 512
# (num-params 1g:21309812 1i: 43447156)

System               1g          1i
WER on dev        38.3      37.6
WER on eval        41.6      40.9
Final train prob      -0.138017 -0.114135
Final valid prob      -0.238659 -0.245208
Final train prob (xent)      -1.66834  -1.47648
Final valid prob (xent)      -2.17419  -2.16365

@danpovey
Copy link
Contributor

nice!

@vijayaditya
Copy link
Contributor Author

vijayaditya commented Nov 28, 2016

Overall tuning of tdnn+lstm recipe gave a 2% abs. reduction in WER.
Compared to 1a, 1i has

  1. 3 TDNN layers between successive LSTM layers, instead of 2
  2. It uses a smaller mini-batch size (128 vs 64) which was found to be helpful in TDNN+LSTMs but not in TDNNs
  3. It uses a lot many more parameters (TDNN output dim 512 vs 1024)
System               1a       1i
WER on dev        38.9      37.6
WER on eval        42.2      40.9
Final train prob      -0.142585 -0.114135
Final valid prob      -0.251197 -0.245208
Final train prob (xent)      -1.73176  -1.47648
Final valid prob (xent)      -2.26965  -2.16365

Edited : Previous comparison was with 1b, but it should have been with 1a.

@naxingyu
Copy link
Contributor

naxingyu commented Nov 29, 2016 via email

@vijayaditya
Copy link
Contributor Author

vijayaditya commented Nov 29, 2016 via email

@danpovey
Copy link
Contributor

danpovey commented Nov 29, 2016 via email

@vijayaditya
Copy link
Contributor Author

@danpovey Is it possible to move Vimal's code or your code as a branch to the main Kaldi repo, so that we can all push our changes to that branch rather than branching from each other's PRs ?

@danpovey
Copy link
Contributor

danpovey commented Nov 29, 2016 via email

swbd : added results for the TDNN recipe which uses the subset-dim
option
@vijayaditya vijayaditya merged commit 4435248 into kaldi-asr:master Nov 30, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants