Hcs keyvals from csv

[will-moore]

See discussion at https://forum.image.sc/t/uploading-key-value-pairs-from-csv-files-into-omero-web-for-plates/60202/27

This should annotate objects from a CSV, following the rules defined there, with plate, well, image columns:

plate           well    image               Cell line
Index.idx.xml                               S2R+        # add KV to plate
Index.idx.xml   B2                          S2R+        # add KV to well only
Index.idx.xml   B2      Well 1 Field 1      S2R+        # add KV to image only
Index.idx.xml           Well 1 Field 2      S2R+        # add KV to image only

To test:

• create a CSV like the one above, with a mixture of rows that have plate, well or image names that match a Plate that you have
• Select the Plate, run the script - You can now pick the csv using the script dialog file-picker (didn't work before)
• The plate, well and image objects that match names in the CSV should be annotated, according to the rules above (1 object per row of the table)

[abhamacher]

Hi Will,

I tested this change and it provides a helpful message. Unfortunately the message is only visible in the detailed log. On level of the "Activities" output there is no exclamation mark or other color hint, that something was not ok with the import. If this is easy to add, it would be nice to add this too, as I expect that most users will not check the detailed log after upload and in case of 384-well plates it might be not easy to even see this output in the then quite long log file.

Overall this state of the script is already a huge improvement! Especially for testing purposes it would be nice to have an updated "Remove all KV-pairs"-script for plates too :-)

Thanks a lot for your effort!
Anna

[JensWendt]

tested it on our live-system once and it worked

[will-moore]

Thanks for the testing.
I need to wait for ome#193 to get merged (to add the annotation scripts to the repo), then I can re-open a PR to the ome repo to add the Plate changes from here and address the warning message etc.

[JensWendt]

May I suggest to alter the paragraph after # read the csv in line 155 or so such as:
# read the csv
file_handle = provider.get_original_file_data(original_file)
delimiter = csv.Sniffer().sniff(file_handle.read(1024)).delimiter
file_handle.close()
print("delimiter is: ",delimiter)
file_handle = provider.get_original_file_data(original_file)
data = list(csv.reader(file_handle, delimiter=delimiter))
file_handle.close()

This would circumvent the problem of varying native delimiters depending on your system configuration.
I tested it with "," and ";" as delimiters and it worked fine.
It might be overly complicated, as I am not quite sure why I have to close the file_handle and then again assign it fresh but that is the result of my trial and error approach.

As I am new to the Github universe I am pretty sure this is the wrong way to propose a code change , but I trust you will put this in the proper workflow if it makes sense.

Jens

[will-moore]

Thanks @JensWendt - I've tried that locally and managed it without having to open the file twice. I set the file_handle back to the start seek(0) so that it reads from the start when reading the whole file.
Also, I used a try/catch since this didn't work on a tiny csv example file I've been using for testing.
So now I have this (which I'll put into a PR soon):

        file_handle = provider.get_original_file_data(original_file)
        try:
            delimiter = csv.Sniffer().sniff(file_handle.read(1024)).delimiter
            print("Using delimiter: ", delimiter)
        except Exception:
            print("Failed to sniff delimiter, using ','")
            delimiter = ","
        # reset to start and read whole file...
        file_handle.seek(0)
        data = list(csv.reader(file_handle, delimiter=delimiter))
        file_handle.close()

[JensWendt]

Heyo,
Yes, that is a better implementation of the idea.
I was not aware that I could have reset the file_handle, good.
The try/catch is definitely the better approach, although in 95% of the cases the script should be used to annotate a large dataset/plate with a big enough .csv to "sniff".
Thank you!