Bug: Curl bug triggered if partially downloaded and can never finish download

[kenyee]

Bug report
If the download fails partway through, a curl bug is triggered and sdkman never manages to download the new JDK, etc. and you see this curl error:

HTTP server doesn't seem to support byte ranges. Cannot resume.

######################################################################## 100.0%curl: (33) HTTP server doesn't seem to support byte ranges. Cannot resume.
16:53:56 
16:53:56 Download has failed, aborting!
16:53:56 
16:53:56 Can not install java 17.0.10-zulu at this time...

To reproduce
Tell sdkman to download a new Zulu JDK image. Kill the process partway through so you get a partial file.
Try doing "sdk env install" again and you should see the above error.
I found this old Stackoverflow issue that links the curl bug:
https://stackoverflow.com/questions/23586214/curl-33-http-server-doesnt-seem-to-support-byte-ranges-cannot-resume

Probably should add a retry or optional retry for the curl?

System info

• OS (e.g. Windows, Linux, Mac, Cygwin, WSL, etc.) and version
  OSX Mac 23.4.0

• Shell and version (e.g. bash --version/zsh --version)
  % zsh --version
  zsh 5.9 (x86_64-apple-darwin23.0)

• The output of sdk version
  SDKMAN!
  script: 5.18.2
  native: 0.1.3

[hgeraldino]

Optional retries won't fix this issue, right? As SDKman uses curl to download the files from some endpoint (candidate mirrors, github, CDNs, etc.), resuming a download will fail if the endpoint doesn't support it.

Does running sdk flush fixes this issue?

[kenyee]

Right, you'd have to delete the sdk before the retry.
Hmm...haven't tried sdk flush. We've been deleting the folder where the downloads happen to work around this.

[helpermethod]

I think cURL's --remove-on-errors may fix this

https://daniel.haxx.se/blog/2022/03/11/remove-leftovers-on-curl-error/

I'll see if I can hack something together.

[kenyee]

Thx...that looks promising. Seems like that should just be the default 😄

"curl’s 246th command line option" 😹

[marc0der]

Hi @kenyee, I've dived into this a bit deeper and have run into a wall with this.

Even though --remove-on-errors might sound like a good idea, it was only introduced curl 7.83.0 not so long ago as you must have read in that blog post. That means that anyone using an older version of curl would break. Since we strive never to harm anyone's user experience, we can't adopt something breaking like this.

How did I discover that it breaks? When I tried to run this in the above experimental branch it failed because the github runner's version of curl didn't have this feature. If I can't even get this to build on CI, I'm certainly not going to roll it out to the greater public.

In the meantime, I suggest using the sdk flush command when this happens occasionally. Please don't delete folders yourself, but use the provided commands to act on the filesystem. Hopefully, this is just an occasional occurrence.

[kenyee]

That's unfortunate...checked that it's 7.87 on MacOSX but Jammy Ubuntu is on 7.81 (https://packages.ubuntu.com/jammy/curl).
Could run "sdk flush" if you see detect a curl error during your download as well and do a retry?

But yes, we'll look into baking in the java version into our jenkins images to prevent this from happening in the future. Thanks for looking at it!

[marc0der]

Retrying feels a bit wrong and I don't think we should go down that route 😛

Anyway, sorry I couldn't be of more help and thanks again for making us aware of it!

[marc0der]

Closing this issue because we can't adopt the curl flag.

[jaceklaskowski]

sdk flush has indeed fixed this issue for me! 🥳