Support for source code archives

[tranzystorekk]

Hi, I recently gained some interest in dra 😄

I was wondering if source code archives that are included in Github releases could also be supported.

Also, there are repos out there that only include release tags, but not GH releases, it would be convenient to be able to select and download source code archives for those too.

[devmatteini]

Hi!

Looking at the GitHub API response we should be able to include source code archives from releases without any problems.

Maybe by default, source code should not be included? Using a flag like --include-source-code would make them available in the assets list.

Also, there are repos out there that only include release tags, but not GH releases, it would be convenient > to be able to select and download source code archives for those too.

I'm not sure about this, so for now let's just implement for the GH releases.

[tranzystorekk]

Maybe by default, source code should not be included?

I'm not entirely opposed to the idea, but I'm not sure what gain you see from filtering those options out by default?
I imagine since they are just urls under the tarball_url and zipball_url response fields, they would be displayed at the bottom of the list as sth like source_tar and source_zip.

[devmatteini]

Yeah, exactly what I was thinking.
There's no real gain in filtering them out other than creating more code to maintain.

[tranzystorekk]

A small rant on Github as I was working on a prototype:

• the archives you get from the Release page in the web UI and from the API are slightly different, i.e. the web UI points to the /release/tags/<tag> while the API gives back the tarball for the specific commit hash, so the file/dir names etc are not what a user might expect
• github notoriously refuses to report back content length for these archives so we'll most probably end up with a special case of a download with unknown/always-empty progress bar

[devmatteini]

That's not good news, but let me know if you need help!

I'm wondering if there is another API to just get the source code for a release?
But from what you said I guess the answer is no.

[tranzystorekk]

Tried to check this, but after redirect the results are (tealdeer example):

• web UI: url https://codeload.github.com/dbrgn/tealdeer/tar.gz/refs/tags/v1.6.0, content-disposition filename=tealdeer-1.6.0.tar.gz
• API: url https://codeload.github.com/dbrgn/tealdeer/legacy.tar.gz/refs/tags/v1.6.0, content-disposition filename=dbrgn-tealdeer-v1.6.0-0-g357f0e7.tar.gz

The API result is the same no matter if we query /repos/:owner/:repo/releases/latest or /repos/:owner/:repo/tags

[tranzystorekk]

Okay, there's a lot of small but disruptive code redesign to be done around this feature, mainly around reading that content-disposition header to determine the filename (but only if the user hasn't specified one via the output flag).

Might require a draft PR for more extensive discussion 🤔

[devmatteini]

At this point let's just use the tarball_url and zipball_url from the release api response.

The problem is that it seems there is no Content-Length HTTP header available.

This will make the progress bar not working like you said.

I don't see any other solution right now.

Have you tried something else?

[tranzystorekk]

I guess the progress bar cant be fixed.

But for the filenames, the tarball urls are usually sth like .../owner/repo/vX.Y.Z, I guess we would have to strip that and append an extension, ie. to end up with "vX.Y.Z.tar.gz"?

[devmatteini]

I think we can just create the filenames manually:

let base_name = format!("{repo}-{tag}");
let tarball_name = format!("{base_name}.tar.gz");
let zipball_name = format!("{base_name}.zip");

This is the same file names used by GitHub if you download the "Source code" asset from the web ui.

[tranzystorekk]

The only problem with that is the archive name doesn't match the dir name that it is unpacked into, but I think we can live with that in the initial implementation 😁