Support platform-specific extensions

[deemp]

As mentioned in #5, the API is non-deterministic without setting the target platform in a query parameter.

@twz123 proposed a way to collect a version - target platform mapping in #5 (comment).

We can add this information to our cache.

For each extension (name, publisher), for each platform, we need an object containing the fields lastUpdated, sha, version. We don't need a url because we can construct it on the fly in Nix.

Our .jsons will be 2+ times larger. I guess, we can for now provide extensions just for Nix-supported platforms.

What should be the mapping from Nix-supported platforms to Marketplace target platforms?

Nix supports:

• Linux i686
• Linux x86_64
• Linux aarch64
• macOS x86_64
• macOS aarch64

VSCode Marketplace supports (excluding win):

• universal
• linux-x64
• linux-arm64
• linux-armhf
• alpine-x64
• alpine-arm64
• darwin-x64
• darwin-arm64
• web

Open VSX supports

• universal
• linux-x64
• linux-arm64
• linux-armhf
• alpine-x64
• alpine-arm64
• darwin-x64
• darwin-arm64
• web

[twz123]

I don't think that platform specific packages will just work as-is on NixOS. I've looked at rust-analyzer linux-x64:

diff --git a/data/cache/vscode-marketplace.json b/data/cache/vscode-marketplace.json
index 2eb7a7e..8a249b0 100644
--- a/data/cache/vscode-marketplace.json
+++ b/data/cache/vscode-marketplace.json
@@ -34516,7 +34516,7 @@
 , {"lastUpdated":"2017-02-01T00:27:58.623Z","name":"markdown-dir","publisher":"russoturisto","sha256":"sha256-c/YEfJfeG7avzKyHLlOQhlePmZLsjWfWjixf/r0o4pQ=","url":"https://russoturisto.gallery.vsassets.io/_apis/public/gallery/publisher/russoturisto/extension/markdown-dir/0.1.0/assetbyname/Microsoft.VisualStudio.Services.VSIXPackage","version":"0.1.0"}
 , {"lastUpdated":"2022-11-21T14:49:50.243Z","name":"russ-monokai-extended","publisher":"russpitcher","sha256":"sha256-NuDjssoQMNEbY9g8WfZVeH4VKgLoKiCO3Cr8mJd4VzE=","url":"https://russpitcher.gallery.vsassets.io/_apis/public/gallery/publisher/russpitcher/extension/russ-monokai-extended/1.0.1/assetbyname/Microsoft.VisualStudio.Services.VSIXPackage","version":"1.0.1"}
 , {"lastUpdated":"2022-11-15T16:02:36.647Z","name":"rust","publisher":"rust-lang","sha256":"sha256-2uV+s//4fAEuibNHjSJop4pdf9KpDHsICDNWmdjJVSs=","url":"https://rust-lang.gallery.vsassets.io/_apis/public/gallery/publisher/rust-lang/extension/rust/0.7.9/assetbyname/Microsoft.VisualStudio.Services.VSIXPackage","version":"0.7.9"}
-, {"lastUpdated":"2023-05-11T00:44:47.053Z","name":"rust-analyzer","publisher":"rust-lang","sha256":"sha256-U1WRHHUm4Yq2u+V6FJcmOmbUpaJ4O4I9lr6vsYwlQZU=","url":"https://rust-lang.gallery.vsassets.io/_apis/public/gallery/publisher/rust-lang/extension/rust-analyzer/0.4.1509/assetbyname/Microsoft.VisualStudio.Services.VSIXPackage","version":"0.4.1509"}
+, {"lastUpdated":"2023-05-11T00:44:47.053Z","name":"rust-analyzer","publisher":"rust-lang","sha256":"sha256-NdQgIPVEEkSQa5Kg4mesydi5nxD0eF6Evf8GHRyyq2g=","url":"https://rust-lang.gallerycdn.vsassets.io/extensions/rust-lang/rust-analyzer/0.4.1509/1683765593921/Microsoft.VisualStudio.Services.VSIXPackage","version":"0.4.1509"}
 , {"lastUpdated":"2022-06-01T20:06:19.093Z","name":"kahatheme","publisher":"rustamkakhramanov","sha256":"sha256-z+d6vx3HlUf9pEilcZ6xzPDkQsIQaqzmoa5Cfr4cVnw=","url":"https://rustamkakhramanov.gallery.vsassets.io/_apis/public/gallery/publisher/rustamkakhramanov/extension/kahatheme/0.1.4/assetbyname/Microsoft.VisualStudio.Services.VSIXPackage","version":"0.1.4"}
 , {"lastUpdated":"2023-03-27T05:41:30.213Z","name":"rust-analyzer-cn","publisher":"rustcc","sha256":"sha256-3STvSEtC0DmvDwzw3l7NiD8RvpqErKCpyVeGCeTLawE=","url":"https://rustcc.gallery.vsassets.io/_apis/public/gallery/publisher/rustcc/extension/rust-analyzer-cn/0.5.3/assetbyname/Microsoft.VisualStudio.Services.VSIXPackage","version":"0.5.3"}
 , {"lastUpdated":"2022-06-24T14:06:41.493Z","name":"night-blossom","publisher":"rustedturnip","sha256":"sha256-VUDHze5YT4052zKMqdzGW2V2Kc+KeY2Cpth9Si4MTYM=","url":"https://rustedturnip.gallery.vsassets.io/_apis/public/gallery/publisher/rustedturnip/extension/night-blossom/0.1.0/assetbyname/Microsoft.VisualStudio.Services.VSIXPackage","version":"0.1.0"}

$ nix build .#extensions.x86_64-linux.vscode-marketplace.rust-lang.rust-analyzer
warning: Git tree '/home/twieczorek/Repos/nix-vscode-extensions' is dirty

$ file result/share/vscode/extensions/rust-lang.rust-analyzer/server/rust-analyzer
result/share/vscode/extensions/rust-lang.rust-analyzer/server/rust-analyzer: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=39f93ace56036127c01532346574e52aaacf4b97, with debug_info, not stripped

$ ldd result/share/vscode/extensions/rust-lang.rust-analyzer/server/rust-analyzer
        linux-vdso.so.1 (0x00007ffc4259e000)
        libgcc_s.so.1 => /nix/store/2j8jqmnd9l7plihhf713yf291c9vyqjm-glibc-2.35-224/lib/libgcc_s.so.1 (0x00007f375e1dc000)
        librt.so.1 => /nix/store/2j8jqmnd9l7plihhf713yf291c9vyqjm-glibc-2.35-224/lib/librt.so.1 (0x00007f375e1d7000)
        libpthread.so.0 => /nix/store/2j8jqmnd9l7plihhf713yf291c9vyqjm-glibc-2.35-224/lib/libpthread.so.0 (0x00007f375e1d2000)
        libm.so.6 => /nix/store/2j8jqmnd9l7plihhf713yf291c9vyqjm-glibc-2.35-224/lib/libm.so.6 (0x00007f375e0f2000)
        libdl.so.2 => /nix/store/2j8jqmnd9l7plihhf713yf291c9vyqjm-glibc-2.35-224/lib/libdl.so.2 (0x00007f375e0ed000)
        libc.so.6 => /nix/store/2j8jqmnd9l7plihhf713yf291c9vyqjm-glibc-2.35-224/lib/libc.so.6 (0x00007f375bff7000)
        /lib64/ld-linux-x86-64.so.2 => /nix/store/2j8jqmnd9l7plihhf713yf291c9vyqjm-glibc-2.35-224/lib64/ld-linux-x86-64.so.2 (0x00007f375e1f8000)

That binary will not work out-of-the box on my system. Either folks need to be instructed to use something like nix-ld, or those extensions need to be massaged (e.g. with patchelf, or by building the extensions from source).

[twz123]

Oh, and for the mappings:

OSX seems obvious:

• macOS x86_64 <=> darwin-x64
• macOS aarch64 <=> darwin-arm64

For Linux, I'd choose the glibc versions:

• Linux i686 <=> 💔
• Linux x86_64l <=> linux-x64
• Linux aarch64 <=> linux-arm64

The musl versions aren't statically linked either (at least rust-analyzer isn't), and with the glibc versions, there's at least nix-ld which may be used to make it work.

Another thing to consider: What about only selecting the extensions that aren't platform specific. I noticed that rust-analyzer is also offered as a version without any binaries (the targetPlatform is simply omitted in the JSON response). That way, folks could bring their own rust-analyzer binary, from nix, or from rustup or from wherever. That works for rust-analyzer. Not sure how this works for other dependencies, and what the expectations are for the users of this flake.

[anilanar]

@twz123 As a NixOS+vscode user for a very long time, I don't remember seeing an extension that doesn't allow specifying a path for whatever binary it was built for/around. It's typical to provide our own binaries to such extensions. So I don't think fixing binaries is responsibility of this repository. That would be very difficult if not impossible.

[twz123]

So I don't think fixing binaries is responsibility of this repository. That would be very difficult if not impossible.

Sure! So would it make sense to focus on those extension versions that don't specify any target platform instead? I'd assume that those wouldn't contain any native libraries/binaries.

[anilanar]

So would it make sense to focus on those extension versions that don't specify any target platform instead?

I'm not sure if I understood your questions correctly. When I use nix-vscode-extension.rust-analyzer, I'd expect to get the latest version that is released for the platform that is most similar to mine (as you specified above) and I'd always provide my own binary path to it using relevant extension setting/env var.

So would it make sense to focus on those extension versions that don't specify any target platform instead?

I fear some extensions will not have platform-independent versions.

I noticed that rust-analyzer is also offered as a version without any binaries

I wonder if that is a mistake they made when releasing that specific version.

[deemp]

@twz123, @anilanar, @AmeerTaweel, and other.

The PR #21 hopefully solves this issue. Please, check.