Use OpenBLAS 0.3.7 that allows USE_LOCKING flag for better multithrea…

[ahmetaa]

…ding support.

[danpovey]

That sounds good-- those crashes have been a nasty headache. Would you mind just running the matrix speed test and make sure there isn't a major change in speed? Hopefully it would only acquire locks for larger operations.

[ahmetaa]

Sorry for the late reply, I will try testing and report back as soon as I can.

[ahmetaa]

@danpovey Ok, I compiled 0.3.7 without and with USE_LOCKING=1 flag and compiled kaldi with --mathlib=OPENBLAS.
Here is the result of matrix-lib-speed-test:

Without USE_LOCKING=1

UnitTestRealFftSpeed,float,512,0.0580108,seconds
UnitTestSplitRadixRealFftSpeed,float,512,0.0238421,seconds
Eig w/o eigenvectors,float,100,0.000236034,seconds
Eig with eigenvectors,float,100,0.00157714,seconds
SVD w/o eigenvectors,float,100,0.000975132,seconds
SVD with eigenvectors,float,100,0.00439906,seconds
Eig w/o eigenvectors,float,150,0.000531912,seconds
Eig with eigenvectors,float,150,0.00355315,seconds
SVD w/o eigenvectors,float,150,0.00175309,seconds
SVD with eigenvectors,float,150,0.00895691,seconds
Eig w/o eigenvectors,float,200,0.000874996,seconds
Eig with eigenvectors,float,200,0.00668097,seconds
SVD w/o eigenvectors,float,200,0.00310087,seconds
SVD with eigenvectors,float,200,0.019289,seconds
Eig w/o eigenvectors,float,300,0.00227785,seconds
Eig with eigenvectors,float,300,0.0212009,seconds
SVD w/o eigenvectors,float,300,0.00852084,seconds
SVD with eigenvectors,float,300,0.0646222,seconds
UnitTestSvdSpeed,float,4,0.148881,seconds
AddMatMat,float,512,0.05442,seconds
AddMatMat,float,1024,0.305483,seconds
UnitTestAddMatMatSpeed,float,2,0.359919,seconds
AddRowSumMat,float,4,0.21226,gigaflops
AddRowSumMat,float,16,0.866852,gigaflops
AddRowSumMat,float,64,5.74437,gigaflops
AddRowSumMat,float,256,15.5886,gigaflops
AddRowSumMat,float,1024,12.1873,gigaflops
UnitTestAddRowSumMatSpeed,float,5,0.125824,seconds
AddColSumMat,float,4,0.356489,gigaflops
AddColSumMat,float,16,1.24677,gigaflops
AddColSumMat,float,64,1.1166,gigaflops
AddColSumMat,float,256,14.2364,gigaflops
AddColSumMat,float,1024,12.5403,gigaflops
UnitTestAddColSumMatSpeed,float,5,0.126287,seconds
AddVecToRows,float,4,0.404032,gigaflops
AddVecToRows,float,16,1.46581,gigaflops
AddVecToRows,float,64,1.71225,gigaflops
AddVecToRows,float,256,10.376,gigaflops
AddVecToRows,float,1024,8.88758,gigaflops
UnitTestAddVecToRowsSpeed,float,5,0.12479,seconds
AddVecToCols,float,4,0.414631,gigaflops
AddVecToCols,float,16,1.49629,gigaflops
AddVecToCols,float,64,1.69468,gigaflops
AddVecToCols,float,256,10.8501,gigaflops
AddVecToCols,float,1024,8.88579,gigaflops
UnitTestAddVecToColsSpeed,float,5,0.124832,seconds
UnitTestRealFftSpeed,double,512,0.0633259,seconds
UnitTestSplitRadixRealFftSpeed,double,512,0.0241392,seconds
Eig w/o eigenvectors,double,100,0.00018096,seconds
Eig with eigenvectors,double,100,0.00219488,seconds
SVD w/o eigenvectors,double,100,0.00129294,seconds
SVD with eigenvectors,double,100,0.00546694,seconds
Eig w/o eigenvectors,double,150,0.000408888,seconds
Eig with eigenvectors,double,150,0.00485301,seconds
SVD w/o eigenvectors,double,150,0.00217009,seconds
SVD with eigenvectors,double,150,0.0124691,seconds
Eig w/o eigenvectors,double,200,0.000822067,seconds
Eig with eigenvectors,double,200,0.010818,seconds
SVD w/o eigenvectors,double,200,0.00431585,seconds
SVD with eigenvectors,double,200,0.0280042,seconds
Eig w/o eigenvectors,double,300,0.00227404,seconds
Eig with eigenvectors,double,300,0.029295,seconds
SVD w/o eigenvectors,double,300,0.0102482,seconds
SVD with eigenvectors,double,300,0.0792789,seconds
UnitTestSvdSpeed,double,4,0.194153,seconds
AddMatMat,double,512,0.071157,seconds
AddMatMat,double,1024,0.501188,seconds
UnitTestAddMatMatSpeed,double,2,0.572362,seconds
AddRowSumMat,double,4,0.220902,gigaflops
AddRowSumMat,double,16,1.42692,gigaflops
AddRowSumMat,double,64,3.58936,gigaflops
AddRowSumMat,double,256,6.73406,gigaflops
AddRowSumMat,double,1024,4.1811,gigaflops
UnitTestAddRowSumMatSpeed,double,5,0.128355,seconds
AddColSumMat,double,4,0.373443,gigaflops
AddColSumMat,double,16,1.5517,gigaflops
AddColSumMat,double,64,1.36798,gigaflops
AddColSumMat,double,256,6.14152,gigaflops
AddColSumMat,double,1024,4.01615,gigaflops
UnitTestAddColSumMatSpeed,double,5,0.130078,seconds
AddVecToRows,double,4,0.365751,gigaflops
AddVecToRows,double,16,1.29103,gigaflops
AddVecToRows,double,64,1.52145,gigaflops
AddVecToRows,double,256,4.31625,gigaflops
AddVecToRows,double,1024,2.68429,gigaflops
UnitTestAddVecToRowsSpeed,double,5,0.129655,seconds
AddVecToCols,double,4,0.374167,gigaflops
AddVecToCols,double,16,1.30918,gigaflops
AddVecToCols,double,64,1.59184,gigaflops
AddVecToCols,double,256,4.64207,gigaflops
AddVecToCols,double,1024,2.67349,gigaflops
UnitTestAddVecToColsSpeed,double,5,0.12724,seconds

With USE_LOCKING=1

UnitTestRealFftSpeed,float,512,0.054673,seconds
UnitTestSplitRadixRealFftSpeed,float,512,0.024579,seconds
Eig w/o eigenvectors,float,100,0.00023818,seconds
Eig with eigenvectors,float,100,0.00158,seconds
SVD w/o eigenvectors,float,100,0.000972986,seconds
SVD with eigenvectors,float,100,0.00384593,seconds
Eig w/o eigenvectors,float,150,0.000597954,seconds
Eig with eigenvectors,float,150,0.00419497,seconds
SVD w/o eigenvectors,float,150,0.00279713,seconds
SVD with eigenvectors,float,150,0.0104659,seconds
Eig w/o eigenvectors,float,200,0.000859976,seconds
Eig with eigenvectors,float,200,0.00675201,seconds
SVD w/o eigenvectors,float,200,0.00315905,seconds
SVD with eigenvectors,float,200,0.018713,seconds
Eig w/o eigenvectors,float,300,0.00229001,seconds
Eig with eigenvectors,float,300,0.017885,seconds
SVD w/o eigenvectors,float,300,0.00745392,seconds
SVD with eigenvectors,float,300,0.058074,seconds
UnitTestSvdSpeed,float,4,0.140158,seconds
AddMatMat,float,512,0.048398,seconds
AddMatMat,float,1024,0.297589,seconds
UnitTestAddMatMatSpeed,float,2,0.346005,seconds
AddRowSumMat,float,4,0.237851,gigaflops
AddRowSumMat,float,16,0.9047,gigaflops
AddRowSumMat,float,64,6.35874,gigaflops
AddRowSumMat,float,256,14.9956,gigaflops
AddRowSumMat,float,1024,11.763,gigaflops
UnitTestAddRowSumMatSpeed,float,5,0.126398,seconds
AddColSumMat,float,4,0.348037,gigaflops
AddColSumMat,float,16,1.17874,gigaflops
AddColSumMat,float,64,1.06727,gigaflops
AddColSumMat,float,256,13.3751,gigaflops
AddColSumMat,float,1024,12.2938,gigaflops
UnitTestAddColSumMatSpeed,float,5,0.126871,seconds
AddVecToRows,float,4,0.391497,gigaflops
AddVecToRows,float,16,1.38372,gigaflops
AddVecToRows,float,64,1.62438,gigaflops
AddVecToRows,float,256,9.45449,gigaflops
AddVecToRows,float,1024,8.33162,gigaflops
UnitTestAddVecToRowsSpeed,float,5,0.126276,seconds
AddVecToCols,float,4,0.392911,gigaflops
AddVecToCols,float,16,1.40706,gigaflops
AddVecToCols,float,64,1.61224,gigaflops
AddVecToCols,float,256,10.4908,gigaflops
AddVecToCols,float,1024,8.73168,gigaflops
UnitTestAddVecToColsSpeed,float,5,0.12528,seconds
UnitTestRealFftSpeed,double,512,0.0629001,seconds
UnitTestSplitRadixRealFftSpeed,double,512,0.0244851,seconds
Eig w/o eigenvectors,double,100,0.00018096,seconds
Eig with eigenvectors,double,100,0.002244,seconds
SVD w/o eigenvectors,double,100,0.00128794,seconds
SVD with eigenvectors,double,100,0.00553107,seconds
Eig w/o eigenvectors,double,150,0.000414848,seconds
Eig with eigenvectors,double,150,0.00479102,seconds
SVD w/o eigenvectors,double,150,0.00304914,seconds
SVD with eigenvectors,double,150,0.0136042,seconds
Eig w/o eigenvectors,double,200,0.000822067,seconds
Eig with eigenvectors,double,200,0.0112109,seconds
SVD w/o eigenvectors,double,200,0.00420713,seconds
SVD with eigenvectors,double,200,0.0282662,seconds
Eig w/o eigenvectors,double,300,0.00232887,seconds
Eig with eigenvectors,double,300,0.0292709,seconds
SVD w/o eigenvectors,double,300,0.0103419,seconds
SVD with eigenvectors,double,300,0.0840769,seconds
UnitTestSvdSpeed,double,4,0.201723,seconds
AddMatMat,double,512,0.076071,seconds
AddMatMat,double,1024,0.502507,seconds
UnitTestAddMatMatSpeed,double,2,0.578601,seconds
AddRowSumMat,double,4,0.226565,gigaflops
AddRowSumMat,double,16,1.48215,gigaflops
AddRowSumMat,double,64,5.03928,gigaflops
AddRowSumMat,double,256,6.82747,gigaflops
AddRowSumMat,double,1024,4.06647,gigaflops
UnitTestAddRowSumMatSpeed,double,5,0.128386,seconds
AddColSumMat,double,4,0.370321,gigaflops
AddColSumMat,double,16,1.42228,gigaflops
AddColSumMat,double,64,1.39679,gigaflops
AddColSumMat,double,256,7.2701,gigaflops
AddColSumMat,double,1024,4.11514,gigaflops
UnitTestAddColSumMatSpeed,double,5,0.126828,seconds
AddVecToRows,double,4,0.379804,gigaflops
AddVecToRows,double,16,1.44896,gigaflops
AddVecToRows,double,64,1.71655,gigaflops
AddVecToRows,double,256,4.77707,gigaflops
AddVecToRows,double,1024,2.70938,gigaflops
UnitTestAddVecToRowsSpeed,double,5,0.126808,seconds
AddVecToCols,double,4,0.36691,gigaflops
AddVecToCols,double,16,1.30389,gigaflops
AddVecToCols,double,64,1.54855,gigaflops
AddVecToCols,double,256,4.27777,gigaflops
AddVecToCols,double,1024,2.65027,gigaflops
UnitTestAddVecToColsSpeed,double,5,0.127431,seconds

Consecutive calls are yielding slightly different results. But seems ok to me.

[danpovey]

@ahmetaa can you please recreate this using a branch name that is not 'master' and make sure any conflicts are addressed? I am scared to merge this because I think my attempt to resolve a conflict via the web interface has somehow confused GitHub and I don't know exactly how it will behave if I merge it now.

[ahmetaa]

@danpovey Sure, will do soon.