Torch-HDF5 Failure during Writing Output by basset_motifs_predict.lua

[dg520]

Hi David,

It seems torch-hdf5 is not very compatible to Basset. The basset_motifs_predict.lua fails to write output. I tested the script line by line inside th interactive mode and encountered the failure at local hdf_out = hdf5.open(opt.out_file, 'w'). Strangely, it didn't fail when read in the hdf5 file (the validation sequences) in the previous steps. The error message is as following:

/usr/dgXXX/torch/install/bin/luajit: /usr/dgXXX/torch/install/share/lua/5.1/hdf5/file.lua:10: HDF5File.__init() requires a fileID - perhaps you want HDF5File.create()?
stack traceback:
	[C]: in function 'assert'
	/usr/dgXXX/torch/install/share/lua/5.1/hdf5/file.lua:10: in function '__init'
	/usr/dgXXX/torch/install/share/lua/5.1/torch/init.lua:91: in function </PHShome/dg520/torch/install/share/lua/5.1/torch/init.lua:87>
	[C]: in function 'open'
	./basset_motifs_predict.lua:57: in main chunk
	[C]: in function 'dofile'
	...gXXX/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
	[C]: at 0x00405810

Actually I met the same error message in the training stage but worked around by reading Deepmind issue #81 and Basset issue #25. I made the following changes:

1. Inside src folder of Basset, I downloaded https://github.com/davek44/torch-hdf5.git and luarocks make the library inside the torch-hdf5 folder.
2. Changed lua/5.1/hdf5/ffi.lua by replacing if maj[0] ~= 1 or min[0] ~= 8 then with if maj[0] ~= 1 or min[0] ~= 10 then, to make it work with HDF5 version 1.10;
3. Changed lua/5.1/hdf5/file.lua by adding fileID=tonumber(fileID) in openFunc and have changed ua/5.1/hdf5/group.lua by replacing .. self._groupID .. with .. tostring(self._groupID) .. in HDF5Group:__tostring(). These two changes overcome the error message for lacking fileID at training stage.

I've also tried to install the hdf5-1.10 version, but then encountered the problem of 2-byte float incompatibility.

I really appreciate if you can help me out in this case. Thanks a lot!

Best,
Dadi

[dg520]

Hi David,

I've made it work by reading Issue #76 from DeepMind. @Cadene 's approach solved my problem. HDF5 should be downgraded to 1.8, instead of the latest 1.10.
Thanks for developing and maintaining the amazing tool Basset.

Best,
Dadi

[Cadene]

Happy to help :D Le mer. 29 août 2018 à 14:15, Dadi <[email protected]> a écrit :

Hi David, I've made it work by reading Issue [#76] (google-deepmind/torch-hdf5#76 <google-deepmind/torch-hdf5#76>) from DeepMind. @Cadene <https://github.com/Cadene> 's approach solved my problem. Thanks for developing and maintaining the amazing tool Basset. Best, Dadi — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#40 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEdvLsJvcVzTdWAIf_fBiNFadOjNa218ks5uVwRpgaJpZM4WR_nI> .

-- Remi Cadene PhD Student LIP6 - UPMC Sorbonne Universities http://remicadene.com 0033635504788

[davek44]

Great, yea the dependency on DeepMind's hdf5 library, which is no longer maintained, is unfortunate. All active development with this project is now happening in TensorFlow here: https://github.com/calico/basenji