Support other git hosting services

[NobodyXu]

IMO it is possible to support for other git hosting by parsing Package::repository, though I think we should straight-up report an error if neither repository nor PkgMeta is specified, since then we cannot find its repository without trusting random people on the internet.

[NobodyXu]

Currently, we use a default template even if Package::repository if not set, that means we could visit the wrong repository owned by somebody else even if the package is not hosted online at all or hosted on platform other than github.

@passcod I think this is a security loophole of cargo-binstall and we should fix this immediately.

[passcod]

Sorry, I can't see what you mean? Can you demo how we'd visit the wrong repo there's no repository set in the cargo meta?

[NobodyXu]

Sorry, I can't see what you mean? Can you demo how we'd visit the wrong repo there's no repository set in the cargo meta?

Oh I checked the source code of gh_crate_meta::Context and that problem is indeed already recognized and fixed.
However it results in panicing instead of a proper error, so I will work on that part after #294 is done.
It's working just fine, I mistook that #[should_panic] means that the Template::render somehow panics.

[passcod]

I think this can be closed?

[NobodyXu]

I think this can be closed?

Let me check the source code again.

[NobodyXu]

I think we might still need some further investigation.

The default url template:

{ repo }/releases/download/v{ version }/{ name }-{ target }-v{ version }.{ archive-format }

is designed for github, I'm not sure whether it would work for bitbucket or gitlab.

Would have to verify that.

[passcod]

Oh, that's true, we could definitely switch the default template for other hosting services

[NobodyXu]

I just created a hello-world project on gitlab and made a release 0.2.0 with the cargo-binstall uploaded as a package.

Gitlab supports two sorts of artifacts:

• Attached files, this is uploaded with a random prefix (check 0.1.0 release I made), so I doubt we can use that
• Provides a url and a link name as the artifacts

According to gitlab doc, there's a permanent link to release assets as long as there's a filepath.

But its seems that filepath cannot be added from the web UI and requires to use its API...

[NobodyXu]

I tried https://gitlab.kitware.com/NobodyXu/hello-world/-/releases/permalink/v0.2.0/downloads/cargo-binstall-x86_64-pc-windows-msvc.zip but doesn't work.

Seems like I have to use its API and cannot just use the link title.

[NobodyXu]

After manually updating the link asset using:

curl --request PUT \
     --data name="cargo-binstall-x86_64-pc-windows-msvc.zip" \
     --data filepath="/binaries/cargo-binstall-x86_64-pc-windows-msvc.zip" \
     --header "PRIVATE-TOKEN: *********" \
     "https://gitlab.kitware.com/api/v4/projects/9564/releases/v0.2.0/assets/links/160"

The permanent asset link finally worked https://gitlab.kitware.com/NobodyXu/hello-world/-/releases/v0.2.0/downloads/binaries/cargo-binstall-x86_64-pc-windows-msvc.zip

@passcod I will use the following template for gitlab:

{ repo }/-/releases/v{ version }/downloads/binaries/{ name }-{ target }.{ archive-format }

The downsides is that maintainers have to manually specify the filepath using the script above or do that in their CI/CD, but I cannot find any other reasonable way to do this.

Gitlab seems to love exposing functionalities in their apis but not their web UI and filepath option is not exposed in
their web UI for creating/updating release.
Other alternative is to create an api client for gitlab and parse something like this.

Does that look good to you?

[NobodyXu]

@passcod I just realized the default url template isn't even suitable for github.

For github, we have links like:

https://github.com/cargo-bins/cargo-binstall/releases/download/v0.12.0/cargo-binstall-aarch64-apple-darwin.zip

So the template should be:

{ repo }/releases/download/v{ version }/{ name }-{ target }.{ archive-format }

[NobodyXu]

Downloading artifacts with bitbucket is much easier.

While BitBucket does not have a release page, it indeeds have a download page.

Using the following link, we can download the artifacts I just uploaded:

https://bitbucket.org/nobodyxusdcdc/hello-world/downloads/cargo-binstall-x86_64-apple-darwin-v0.12.0.zip

So the template for bitbucket should be:

{ repo }/downloads/{ name }-{ target }-v{ version }.{ archive-format }

@passcod Does this LGTY?

[passcod]

{ repo }/releases/download/v{ version }/{ name }-{ target }.{ archive-format }

our own template (above) is different from the github default, but the default one should remain the same. it's not uncommon to include the version in the package name, we just don't do it in binstall so that the /latest/ download URLs can be used for that first install.

[passcod]

One thought with the gitlab thing is that we can't really tell if something is using gitlab just from the URL. We can make a reasonable guess, but idk if that's good enough.

[passcod]

bitbucket looks good :)

[NobodyXu]

One thought with the gitlab thing is that we can't really tell if something is using gitlab just from the URL. We can make a reasonable guess, but idk if that's good enough.

That's right.

I also realized it is also true for github, it can have enterprise version with custom domain name such as https://github.sydney.edu.au/ .

[passcod]

Hmm let's make a reasonable guess (for both) and go with the template you proposed, plus an extra section in SUPPORT.md for gitlab support.

[NobodyXu]

Finally, there's sourceforge, a popular code hosting platform for opensource projects despite its ugly web UI.

I think we can use the following template for it:

{ repo }/files/binaries/v{ version }/{ name }-{ target }.{ archive-format }/download

{ repo }/files is the "download page" of sourceforge.
binaries/v{ version } is the directory and { name }-{ target }.{ archive-format } is the filename.

@passcod Does that look good to you?

[NobodyXu]

Actually, we can use the hello-world api to discover its hosting services.

For github, we use:

curl https://api.github.com/zen

for github.com and

curl https://[hostname]/api/v3/zen

for github enterprise server, though it might require access token.

For gitlab, we can use the metadata API, but it requires a private token so it might not work.

[NobodyXu]

though it might require access token.

Perhaps we should also add a new cmdline option -H/--header to pass that auth tokens.

[NobodyXu]

@ryankurte What's your thought on this?