finish the script to transform matrix to png image for debugging

[YiwenShaoStephen]

No description provided.

[danpovey]

@hhadian, can you please test this out? You can use a command like

copy-matrix --binary=false foo.ark:56431 -

to extract a matrix from an images.scp file in text form, as I believe the python script requires.
Extracting the matrices from the 'egs' (required to test the image augmentation binary) will require doing something like

nnet3-copy-egs ark:egs.1.ark ark,t:- | head -n 1000 > foo.txt

and searching for open-square-bracket in foo.txt in an editor and extracting the matrix manually (or writing a script to do the same).
You can then try inserting nnet3-egs-augment-image into the pipe and doing the same thing.

[hhadian]

Will do it.

[hhadian]

@YiwenShaoStephen, I get this error when trying the script with a matrix:

Traceback (most recent call last):
  File "image/matrix_to_image.py", line 50, in <module>
    if tmp[-1] == ']': # drop the "]" in the last row
IndexError: list index out of range

I guess the if before this if removes the only item in the list. Please note that a Kaldi matrix in text format can start with a line with a single [ (and nothing else) and then the first row on the next line:

[
  7 4 ...
...
... ]

Also I think it might be better to rename this variable to something other than tmp (maybe line or row)

[danpovey]

Hossein, you can go ahead and fix these issues if you want- he is in a very different time zone.

…

On Tue, Apr 25, 2017 at 8:06 PM, Hossein Hadian ***@***.***> wrote: @YiwenShaoStephen <https://github.com/YiwenShaoStephen>, I get this error when trying the script with a matrix: Traceback (most recent call last): File "image/matrix_to_image.py", line 50, in <module> if tmp[-1] == ']': # drop the "]" in the last row IndexError: list index out of range I guess the if before this if removes the only item in the list. Please note that a Kaldi matrix in text format can start with a line with a single [ (and nothing else) and then the first row on the next line: [ 7 4 ... ... ... ] Also I think it might be better to rename this variable to something other than tmp (maybe line or row) — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#1575 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADJVuxyWHLold3I0KzXBWY8uWcoqjJHDks5rzoqFgaJpZM4NHjL0> .

[hhadian]

I would, but it seems the script has more issues, and even after a temporary fix the png image that was produced could not be opened. I guess he should take a look and test it first.

[YiwenShaoStephen]

I have just seen your conversation. What is the problem now? I thought the matrix should have a fixed format as it is in the example. And I also thought the value in the matrix should be a float between 0 to 1, but it seems not to be like this in your above discussion. Could you provide me with more detailed information about the matrix? Or the error information? Maybe I can fix them immediately.

[hhadian]

Try preparing cifar data (using local/prepare_data.sh) and then try copy-matrix --binary=false data/cifar10_train/data/images.ark:6 - | image/matrix_to_image.py --color 3 > foo.png
Also please try this on CLSP grid to make sure PIL is working as you expect

[danpovey]

He doesn't have an account here.

…

On Tue, Apr 25, 2017 at 10:58 PM, Hossein Hadian ***@***.***> wrote: Try preparing cifar data (using local/prepare_data.sh) and then try copy-matrix --binary=false data/cifar10_train/data/images.ark:6 - | image/matrix_to_image.py --color 3 > foo.png Also please try this on CLSP grid to make sure PIL is working as you expect — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#1575 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADJVu5Hw2ZgsM9JOWVOVXtlm2EQ0ftZRks5rzrLJgaJpZM4NHjL0> .

[hhadian]

The values I used in the example were just for demonstration. You can assume they are in the range [0,1].

[hhadian]

He doesn't have an account here.

Oh! So I will try it locally with the latest version of PIL.

[YiwenShaoStephen]

Install the PILLOW (fork version of PIL) or it may cause troubles in image saving.

[YiwenShaoStephen]

Or you can provide me with one test matrix that you can't generate a png file with. It will help me better know the current problem.

[danpovey]

The aim is to get this working without any special extra installation of python modules. This is a very small and (relatively) unimportant script, and we would not add a new thing to the list of things that kaldi depends on, just for this. Surely there must be *some* image format that standard PIL is capable of saving, or what use would the library be?

…

On Tue, Apr 25, 2017 at 11:06 PM, YiwenShaoStephen ***@***.*** > wrote: Or you can provide me with one test matrix that you can't generate a png file with. It will help me better know the current problem. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#1575 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADJVu5wir-xebVQWs4abG197iLoTvW7Fks5rzrStgaJpZM4NHjL0> .

[YiwenShaoStephen]

The standard PIL needs prerequisite libraries to be built for different image format. And I met a same trouble when installing these libraries as this one: http://stackoverflow.com/questions/20753607/while-upgrading-python-imaging-library-pil-it-tells-me-jpeg-support-not-avai.

[hhadian]

Or you can provide me with one test matrix that you can't generate a png file with. It will help me better know the current problem.

I already told you how to get a test matrix (i.e. by piping output of copy-matrix)

I checked the installed PIL on the grid: it is actually Pillow v2.6.1.
Which is more common? Pillow or PIL? It's odd that they have the same module name (i.e. PIL)

[danpovey]

I would be OK with matplotlib. Apparently that can display images too, and it may have less version compatibility problems.

…

On Tue, Apr 25, 2017 at 11:21 PM, Hossein Hadian ***@***.***> wrote: Or you can provide me with one test matrix that you can't generate a png file with. It will help me better know the current problem. I already told you how to get a test matrix (i.e. by piping output of copy-matrix) I checked the installed PIL on the grid: it is actually Pillow v2.6.1. Which is more common? Pillow or PIL? It's odd that they have the same module name (i.e. PIL) — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#1575 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADJVu-Dit9z_iWTj8VddA8NFspsOAlsXks5rzrgngaJpZM4NHjL0> .

[YiwenShaoStephen]

Oh, that would be great. Actually the Pillow is more common today. And when people talk about PIL, they usually refer to Pillow.

[hhadian]

BTW, have you considered simple formats like bitmap too? AFAIK bitmap is so simple you can write it without using any library.

[YiwenShaoStephen]

Yes, I've tried. BMP image is also available. Just change the "png" in im.save() to "bmp".

[YiwenShaoStephen]

Oh, I get you wrong. I haven't tried to write bmp myself. I'm not familiar with the image format. If it is necessary to run without libraries, I can learn to write it.

[hhadian]

Another option is to use matplotlib (which is also used in Kaldi's reporting scripts) to save a color figure (I assume we can use imshow or imagesc IIRC and then save the resulting figure).

[danpovey]

First try to get it working by by making sure it works using actual Kaldi matrices, like ``` [ 0 0.1 0.2 0.3 0.4 0.5 ] ``` It would be nice to do this library-free, but you can do that after we get your current version working.

…

On Tue, Apr 25, 2017 at 11:38 PM, YiwenShaoStephen ***@***.*** > wrote: Oh, I get you wrong. I haven't tried to write bmp myself. I'm not familiar with the image format. If it is necessary to run without libraries, I can learn to write it. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#1575 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADJVuy5rD-jpy_EfpGmqToFYQNwZKHsYks5rzrxHgaJpZM4NHjL0> .

[YiwenShaoStephen]

Good news! BTW, from the above discussion, I notice that the input matrix should come from the process_data.sh right? I look into its codes and find that the output matrix doesn't share a same format as the example matrix in the matrix_to_image.py's comments. Besides, the row index and column index doesn't exchange either. etc. the row index still refers to the height. Do I miss or misunderstand something?

[danpovey]

Good news! BTW, from the above discussion, I notice that the input matrix should come from the process_data.sh right?

No, they won't come from there. They will be printed out by Kaldi programs. The normal format will be [ 0.0 0.1 0.2 0.3 0.4 0.5 ] but ideally you should be tolerant of differently-place square brackets, i.e. accept the same type of matrices that matlab would accept. I look into its codes and find that the output matrix doesn't share a same format as the example matrix in the matrix_to_image.py's comments. Besides, the row index and column index doesn't exchange either. etc. the row index still refers to the height. If that is the case, it's a bug. Hossein, see if that's the case. The row-index should be the width.

…

Do I miss or misunderstand something? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#1575 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADJVu5FDXCP9BIy1qMe9Ukpcj-TAquHDks5rzsEkgaJpZM4NHjL0> .

[hhadian]

The matrix that is written is such that the number of its rows equals the height if the image (i.e. each row of the matrix pertains to each row of the image). I see this is not as the comments say. I can fix it.
Anyway, this can't cause the produced image to be unopenable.

[YiwenShaoStephen]

I've fixed the bugs on failing to read different matrix format and made a new PR to kaldi_52.

[hhadian]

I looked into foo.png (i.e. produced image) and it was all in text format (listing the values in a special order). I guess the version of Pillow I have is old, I'll try to update it.