Download button for output files disappearing

[SergeWielhouwer]

Hi,

I am currently experiencing issues downloading output files. Sometimes the download button appears, but most of the time it is not visible.

My config file looks like this;

Everytime I execute the script, the correct file is placed in temporary directory (resultFiles) on our server running script-server. It seems to detect the correct file(s), but the download button is not shown at the bottom of the script output.

The last ten lines in the server log are as follows;
2022-12-01 15:52:56,907 [script_server.execution_service.INFO] Calling script #498: bash /mnt/shared/tools/script-server/GS/run_ffpe_qc.sh 105285 Human forward 8
2022-12-01 15:52:56,917 [script_server.file_download_feature.INFO] Created download folder for s.wielhouwer: temp/resultFiles/18e344481b61/1669906376917
2022-12-01 15:52:56,919 [tornado.access.INFO] 200 POST /executions/start (10.159.1.224) 33.86ms
2022-12-01 15:52:56,928 [tornado.access.INFO] 101 GET /executions/io/498 (10.159.1.224) 1.25ms
2022-12-01 15:52:56,930 [script_server.audit_utils.WARNING] Could not get hostname for 10.159.1.224
2022-12-01 15:52:56,931 [script_server.audit_utils.WARNING] Could not get hostname for 10.159.1.224
2022-12-01 15:52:57,031 [script_server.audit_utils.WARNING] Could not get hostname for 10.159.1.224
2022-12-01 15:52:57,032 [web_server.INFO] s.wielhouwer disconnected
2022-12-01 15:52:57,036 [script_server.audit_utils.WARNING] Could not get hostname for 10.159.1.224
2022-12-01 15:52:57,037 [tornado.access.INFO] 200 POST /executions/cleanup/498 (10.159.1.224) 2.13ms

Any ideas what could possibly cause this? Is this related to the hostname error?

Best regards,

Serge

[bugy]

Hi @SergeWielhouwer do you see any errors in the browser log, when it happens?
Also, may be you could debug it on a network tab in browser, if you open a websocket for the process, may be you can check and send me the messages, which are sent over it?

[SergeWielhouwer]

Hello @bugy, thank you for your quick reply.
I am not sure if I see any errors in the browser log (where do I see this in Firefox), I only checked the network tab.

For a successful run (Firefox). It looks like this (I've hid the left side on purpose);

Server.log (Currently on VPN to internal network where server is located);
2022-12-02 09:35:35,100 [web_server.INFO] Calling script FFPE QC. User {'auth_username': 's.wielhouwer', 'ip': ' external IP address}
2022-12-02 09:35:35,102 [script_server.execution_service.INFO] Calling script #516: bash /mnt/shared/tools/script-server/GS/run_ffpe_qc.sh --project 105285 -c Human -s forward --umi_length 8
2022-12-02 09:35:35,110 [script_server.file_download_feature.INFO] Created download folder for s.wielhouwer: temp/resultFiles/18e344481b61/1669970135111
2022-12-02 09:35:35,112 [tornado.access.INFO] 200 POST /executions/start (external IP address) 29.07ms
2022-12-02 09:35:35,150 [tornado.access.INFO] 101 GET /executions/io/516 (external IP address) 1.14ms
2022-12-02 09:35:35,152 [script_server.audit_utils.WARNING] Could not get hostname for (external IP address)
2022-12-02 09:35:35,152 [script_server.audit_utils.WARNING] Could not get hostname for (external IP address)
2022-12-02 09:35:40,257 [script_server.audit_utils.WARNING] Could not get hostname for (external IP address)
2022-12-02 09:35:40,257 [web_server.INFO] s.wielhouwer disconnected
2022-12-02 09:35:40,280 [script_server.audit_utils.WARNING] Could not get hostname for (external IP address)
2022-12-02 09:35:40,280 [tornado.access.INFO] 200 POST /executions/cleanup/516 (external IP address) 2.02ms

I am unfortunately not familiar with websockets. If they are easy to set up, please let me know. The button(s)now seems to appear every time I click execute (maybe one time it didnt).

The main thing is seeing the download button, does this also have to do with any GET/POST requests? Or does a GET request only happen if I click that button?

Thank you in advance.

[bugy]

Hi, @SergeWielhouwer thanks for reply!
On the screenshot:
The first GET is for websocket (101 status means protocol exchange, from HTTP to websockets), if you could try to open details for it and see info (it should show all the sent messages via the socket), that would be great
The second GET is favicon, not relevant here.

[SergeWielhouwer]

Thanks for the clear explanation.

I hope the following images are relevant (this is for a successful run), if not please let me know.

I will let someone who is on the internal LAN of scriptserver try it again to see if he can see the download buttons still.

[SergeWielhouwer]

One of my colleagues is also able to see the download buttons on the bottom. It seems that if I remove the sleep 5 and immediately let the script exit 0 then the buttons disappear.

[bugy]

@SergeWielhouwer thanks a lot! Could you check the websocket messages, please, when there are no download buttons?
I need to know, if they are not shown because of UI problem, or the server is not sending them at all

Also, if you add a sleep, does it always show the buttons?

[SergeWielhouwer]

Thanks for the feedback @bugy. Yes if I add sleep 5 && exit 0 then it will always show the buttons as expected (me and my colleagues clicked quite often on execute).

No sleep, directly exit 0

Sleep 5, then exit 0

[bugy]

I see, thanks!
Unfortunately I won't be able to fix it in the nearest days, so I would suggest you to ask some short sleep (I believe 1 sec will be enough) to scripts with download :(

[SergeWielhouwer]

Thank you very much for your help! At least we can still use the download functionality, which is great.
Adding a sleep is no problem, I expect all our pipelines to atleast run for a few seconds anyways :)

Btw Is there somewhere where I can read about the available types for output_files? I am wondering if changing the type to something else (e.g., inline_image?) changes how the output file gets processed.

[bugy]

@SergeWielhouwer there are only 2 types, default one for download and inline image. They should work in similar way

[SergeWielhouwer]

One small question, if I want to find multiple report HTML files in a directory. Why does the following not work?

2022-12-02 13:06:29,918 [script_server.file_download_feature.WARNING] Couldn't find file for /mnt/scratch/script-server/${PARAM_GS_PROJECT_NUMBER}/ffpe_qc/plots/*.html

Hardcoding it to /mnt/scratch/script-server/104326-158/ffpe_qc/plots/*.html does work and in the script echo ${PARAM_GS_PROJECT_NUMBER} does show the expected 104326-158 value. But it looks like it doesn't resolve the bash variable? Am I using the config incorrectly?

[bugy]

Hi @SergeWielhouwer you have to use parameter name, not environment variable name, i.e. ${GS project number} in this case

[SergeWielhouwer]

Hi @bugy . Thanks! It worked :)

[Testmann]

Hello All.

First thanks to @SergeWielhouwer for showing that sleep can be used as a workaround.
I've encountered this problem today while playing around with the download file feature.

The problem can be reproduced (on linux) with any script that calls a subshell - and fixed with the sleep 1 && exit 0

Script and Conf-File below.
The downloadable-file helloworld.txt is just a simple static textfile containing "Hello World".
No errors/warnings in the server-log, when executing this.

I hope this can help to find the cause of the problem.

Behaviour:
No Parameter given: Download Button
Parameter given: No Download Button

#!/bin/bash
#hello_subshell.sh
if [ -n "$1" ] ; then  
    echo "$( uptime )"  
else  
    echo "Hello World"  
fi

{
	"name": "Hello_Subshell"
	"script_path": "hello_subshell.sh"
	"working_directory": "/tmp"
	"description": "If no parameter is used, echoes \"Hello World\" else calls \"uptime\" via a subshell",
	"group": "",
	"parameters": [
		{
			"name": "Enter value for subshell"
			"type": "text"
			"description": "Enter value for subshell"
		},
	],
	"output_files": [
		"/tmp/helloworld.txt"
	]
}

[bugy]

I think I found the root cause for it, it will be fixed in the next release