Skip to content

Commit

Permalink
Initial Linux nightly bundles upload (#8913)
Browse files Browse the repository at this point in the history
Changes Zed CI to build and upload Linux nightly bundles.

* `todo!(linux)` are replaced with `TODO linux` to make `todo!`-based
workflows more convenient
* renames `run-build-dmg` label into `run-bundling`, also renames a few
GH Actions entries to be more generic
* make another upload path for Linux, which keeps a separate file with SHA to version the nightly artifact.
* adds a `*.deb` package building with a couple of caveats, marked with
new `TODO linux` entries:

1. `cargo-bundle` is not very flexible, so it generates artifacts with
the structure and names that we're unable to alter before/during the
generation.
For that, a set of extra steps is made by repacking the *.deb package —
this is not very portable between different Linux distros, so later one
needs to find a way to combine multiple package types in this script.

2. `cargo-bundle` is not able to properly generate the *.msi bundle
despite declaring it in the features:
burtonageo/cargo-bundle#116
Windows needs to invent its own way of bundling or fix the tool.

3. Both `cli` and `zed` binaries are added into the archive under
`/usr/local/bin/` path with their `-$channel` suffix
(-nightly/-preview/-dev/-stable) and a `/usr/local/bin/zed ->
/usr/local/bin/cli-nightly` symlink is made to make CLI work as Zed
launcher:
```
~/work/zed kb/linux-nightly:origin/kb/linux-nightly*​ ❯ dpkg -c target/zed_amd64.deb 
drwxr-xr-x allaptop/allaptop 0 2024-03-06 00:53 ./
drwxr-xr-x allaptop/allaptop 0 2024-03-06 00:53 ./usr/
drwxr-xr-x allaptop/allaptop 0 2024-03-06 00:53 ./usr/local/
drwxr-xr-x allaptop/allaptop 0 2024-03-06 00:53 ./usr/local/bin/
-rwxr-xr-x allaptop/allaptop 8746832 2024-03-06 00:53 ./usr/local/bin/cli-nightly
-rwxr-xr-x allaptop/allaptop 689078560 2024-03-06 00:53 ./usr/local/bin/zed-nightly
drwxr-xr-x allaptop/allaptop         0 2024-03-06 00:53 ./usr/share/
drwxr-xr-x allaptop/allaptop         0 2024-03-06 00:53 ./usr/share/applications/
-rw-r--r-- allaptop/allaptop       153 2024-03-06 00:53 ./usr/share/applications/zed.desktop
drwxr-xr-x allaptop/allaptop         0 2024-03-06 00:53 ./usr/share/icons/
drwxr-xr-x allaptop/allaptop         0 2024-03-06 00:53 ./usr/share/icons/hicolor/
drwxr-xr-x allaptop/allaptop         0 2024-03-06 00:53 ./usr/share/icons/hicolor/1024x1024@2x/
drwxr-xr-x allaptop/allaptop         0 2024-03-06 00:53 ./usr/share/icons/hicolor/1024x1024@2x/apps/
-rw-r--r-- allaptop/allaptop    716288 2024-03-06 00:53 ./usr/share/icons/hicolor/1024x1024@2x/apps/zed.png
drwxr-xr-x allaptop/allaptop         0 2024-03-06 00:53 ./usr/share/icons/hicolor/512x512/
drwxr-xr-x allaptop/allaptop         0 2024-03-06 00:53 ./usr/share/icons/hicolor/512x512/apps/
-rw-r--r-- allaptop/allaptop    239870 2024-03-06 00:53 ./usr/share/icons/hicolor/512x512/apps/zed.png
lrwxrwxrwx allaptop/allaptop         0 2024-03-06 00:53 ./usr/local/bin/zed -> /usr/local/bin/cli-nightly
```

But the CLI does not work under Linux yet and there's no way to install
that CLI from Zed now; Zed binary itself is not able to open
`file/location:12:34`-like things and set up the env properly, but is
able to start or open a directory.

So, this structure can be considered temporary and changed, if needed.

4. Zed Nightly on Linux does not know how to update itself, so all
nightly publishing is not picked up automatically.

5. Rust cache from `main` builds does not get shared between CI jobs,
due to being run in a different CI job that forms a different CI key, so
```
      - name: Cache dependencies
        uses: swatinem/rust-cache@v2
        with:
          save-if: ${{ false }}
```
would not work.
This makes Linux bundling jobs long.

Release Notes:

- N/A
  • Loading branch information
SomeoneToIgnore authored Mar 7, 2024
1 parent e85f190 commit ffebe2e
Show file tree
Hide file tree
Showing 9 changed files with 295 additions and 33 deletions.
92 changes: 85 additions & 7 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -152,12 +152,12 @@ jobs:
- name: Build Zed
run: cargo build -p zed

bundle:
name: Bundle macOS app
bundle-mac:
name: Create a macOS bundle
runs-on:
- self-hosted
- bundle
if: ${{ startsWith(github.ref, 'refs/tags/v') || contains(github.event.pull_request.labels.*.name, 'run-build-dmg') }}
if: ${{ startsWith(github.ref, 'refs/tags/v') || contains(github.event.pull_request.labels.*.name, 'run-bundling') }}
needs: [macos_tests]
env:
MACOS_CERTIFICATE: ${{ secrets.MACOS_CERTIFICATE }}
Expand Down Expand Up @@ -212,12 +212,12 @@ jobs:
- name: Generate license file
run: script/generate-licenses

- name: Create app bundle
run: script/bundle
- name: Create macOS app bundle
run: script/bundle-mac

- name: Upload app bundle to workflow run if main branch or specific label
uses: actions/upload-artifact@v3
if: ${{ github.ref == 'refs/heads/main' }} || contains(github.event.pull_request.labels.*.name, 'run-build-dmg') }}
uses: actions/upload-artifact@v4
if: ${{ github.ref == 'refs/heads/main' }} || contains(github.event.pull_request.labels.*.name, 'run-bundling') }}
with:
name: Zed_${{ github.event.pull_request.head.sha || github.sha }}.dmg
path: target/release/Zed.dmg
Expand All @@ -232,3 +232,81 @@ jobs:
body: ""
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

bundle-deb:
name: Create a *.deb Linux bundle
runs-on: ubuntu-22.04 # keep the version fixed to avoid libc and dynamic linked library issues
if: ${{ startsWith(github.ref, 'refs/tags/v') || contains(github.event.pull_request.labels.*.name, 'run-bundling') }}
needs: [linux_tests]
env:
ZED_CLIENT_CHECKSUM_SEED: ${{ secrets.ZED_CLIENT_CHECKSUM_SEED }}
DIGITALOCEAN_SPACES_ACCESS_KEY: ${{ secrets.DIGITALOCEAN_SPACES_ACCESS_KEY }}
DIGITALOCEAN_SPACES_SECRET_KEY: ${{ secrets.DIGITALOCEAN_SPACES_SECRET_KEY }}
steps:
- name: Checkout repo
uses: actions/checkout@v4
with:
clean: false
submodules: "recursive"

- name: Cache dependencies
uses: swatinem/rust-cache@v2
with:
save-if: ${{ github.ref == 'refs/heads/main' }}

- name: Configure linux
shell: bash -euxo pipefail {0}
run: script/linux

- name: Determine version and release channel
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
run: |
set -eu
version=$(script/get-crate-version zed)
channel=$(cat crates/zed/RELEASE_CHANNEL)
echo "Publishing version: ${version} on release channel ${channel}"
echo "RELEASE_CHANNEL=${channel}" >> $GITHUB_ENV
expected_tag_name=""
case ${channel} in
stable)
expected_tag_name="v${version}";;
preview)
expected_tag_name="v${version}-pre";;
nightly)
expected_tag_name="v${version}-nightly";;
*)
echo "can't publish a release on channel ${channel}"
exit 1;;
esac
if [[ $GITHUB_REF_NAME != $expected_tag_name ]]; then
echo "invalid release tag ${GITHUB_REF_NAME}. expected ${expected_tag_name}"
exit 1
fi
# TODO linux : Find a way to add licenses to the final bundle
# - name: Generate license file
# run: script/generate-licenses

- name: Create Linux *.deb bundle
run: script/bundle-linux

- name: Upload app bundle to workflow run if main branch or specific label
uses: actions/upload-artifact@v4
if: ${{ github.ref == 'refs/heads/main' }} || contains(github.event.pull_request.labels.*.name, 'run-bundling') }}
with:
name: Zed_${{ github.event.pull_request.head.sha || github.sha }}.deb
path: target/release/*.deb

# TODO linux : make it stable enough to be uploaded as a release
# - uses: softprops/action-gh-release@v1
# name: Upload app bundle to release
# if: ${{ env.RELEASE_CHANNEL == 'preview' || env.RELEASE_CHANNEL == 'stable' }}
# with:
# draft: true
# prerelease: ${{ env.RELEASE_CHANNEL == 'preview' }}
# files: target/release/Zed.dmg
# body: ""
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
55 changes: 47 additions & 8 deletions .github/workflows/release_nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,8 @@ jobs:
- name: Run tests
uses: ./.github/actions/run_tests

bundle:
name: Bundle app
bundle-mac:
name: Create a macOS bundle
if: github.repository_owner == 'zed-industries'
runs-on:
- self-hosted
Expand All @@ -77,9 +77,6 @@ jobs:
clean: false
submodules: "recursive"

- name: Limit target directory size
run: script/clear-target-dir-if-larger-than 100

- name: Set release channel to nightly
run: |
set -eu
Expand All @@ -90,8 +87,50 @@ jobs:
- name: Generate license file
run: script/generate-licenses

- name: Create app bundle
run: script/bundle
- name: Create macOS app bundle
run: script/bundle-mac

- name: Upload Zed Nightly
run: script/upload-nightly macos

bundle-deb:
name: Create a *.deb Linux bundle
if: github.repository_owner == 'zed-industries'
runs-on: ubuntu-22.04 # keep the version fixed to avoid libc and dynamic linked library issues
needs: tests
env:
DIGITALOCEAN_SPACES_ACCESS_KEY: ${{ secrets.DIGITALOCEAN_SPACES_ACCESS_KEY }}
DIGITALOCEAN_SPACES_SECRET_KEY: ${{ secrets.DIGITALOCEAN_SPACES_SECRET_KEY }}
ZED_CLIENT_CHECKSUM_SEED: ${{ secrets.ZED_CLIENT_CHECKSUM_SEED }}
steps:
- name: Checkout repo
uses: actions/checkout@v4
with:
clean: false
submodules: "recursive"

- name: Cache dependencies
uses: swatinem/rust-cache@v2
with:
save-if: ${{ github.ref == 'refs/heads/main' }}

- name: Configure linux
shell: bash -euxo pipefail {0}
run: script/linux

- name: Set release channel to nightly
run: |
set -euo pipefail
version=$(git rev-parse --short HEAD)
echo "Publishing version: ${version} on release channel nightly"
echo "nightly" > crates/zed/RELEASE_CHANNEL
# TODO linux : find a way to add licenses to the final bundle
# - name: Generate license file
# run: script/generate-licenses

- name: Create Linux *.deb bundle
run: script/bundle-linux

- name: Upload Zed Nightly
run: script/upload-nightly
run: script/upload-nightly linux-deb
6 changes: 3 additions & 3 deletions crates/gpui/src/platform/linux/platform.rs
Original file line number Diff line number Diff line change
Expand Up @@ -324,7 +324,7 @@ impl Platform for LinuxPlatform {
})
}

//todo!(linux)
//TODO linux
fn app_path(&self) -> Result<PathBuf> {
Err(anyhow::Error::msg(
"Platform<LinuxPlatform>::app_path is not implemented yet",
Expand All @@ -338,7 +338,7 @@ impl Platform for LinuxPlatform {
UtcOffset::UTC
}

//todo!(linux)
//TODO linux
fn path_for_auxiliary_executable(&self, name: &str) -> Result<PathBuf> {
Err(anyhow::Error::msg(
"Platform<LinuxPlatform>::path_for_auxiliary_executable is not implemented yet",
Expand Down Expand Up @@ -390,7 +390,7 @@ impl Platform for LinuxPlatform {
})
}

//todo!(linux): add trait methods for accessing the primary selection
//TODO linux: add trait methods for accessing the primary selection

fn read_credentials(&self, url: &str) -> Task<Result<Option<(String, Vec<u8>)>>> {
let url = url.to_string();
Expand Down
2 changes: 1 addition & 1 deletion crates/gpui/src/platform/linux/wayland/client.rs
Original file line number Diff line number Diff line change
Expand Up @@ -281,7 +281,7 @@ impl Client for WaylandClient {
CursorStyle::ResizeUp => "n-resize".to_string(),
CursorStyle::ResizeDown => "s-resize".to_string(),
CursorStyle::ResizeUpDown => "ns-resize".to_string(),
CursorStyle::DisappearingItem => "grabbing".to_string(), // todo!(linux) - couldn't find equivalent icon in linux
CursorStyle::DisappearingItem => "grabbing".to_string(), // TODO linux - couldn't find equivalent icon in linux
CursorStyle::IBeamCursorForVerticalLayout => "vertical-text".to_string(),
CursorStyle::OperationNotAllowed => "not-allowed".to_string(),
CursorStyle::DragLink => "dnd-link".to_string(),
Expand Down
2 changes: 1 addition & 1 deletion crates/gpui/src/platform/linux/x11/client.rs
Original file line number Diff line number Diff line change
Expand Up @@ -361,7 +361,7 @@ impl Client for X11Client {
Box::new(X11Window(window_ptr))
}

//todo!(linux)
// TODO linux
fn set_cursor_style(&self, _style: CursorStyle) {}

fn get_clipboard(&self) -> Rc<RefCell<dyn ClipboardProvider>> {
Expand Down
1 change: 1 addition & 0 deletions crates/zed/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ name = "zed"
version = "0.127.0"
publish = false
license = "GPL-3.0-or-later"
authors = ["Zed Team <[email protected]>"]

[lints]
workspace = true
Expand Down
106 changes: 106 additions & 0 deletions script/bundle-linux
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
#!/usr/bin/env bash

set -euxo pipefail

build_flag="--release"
target_dir="release"
bundle_name=""
zed_crate="zed"


help_info() {
echo "
Usage: ${0##*/} [options] [bundle_name]
Build the application bundle for Linux.
Options:
-d Compile in debug mode
-h Display this help and exit
"
}

while getopts 'dh' flag
do
case "${flag}" in
d)
export CARGO_INCREMENTAL=true
export CARGO_BUNDLE_SKIP_BUILD=true
build_flag="";
target_dir="debug"
;;
h)
help_info
exit 0
;;
esac
done

shift $((OPTIND-1))

if [[ $# -gt 0 ]]; then
if [ "$1" ]; then
bundle_name=$1
fi
fi

export ZED_BUNDLE=true

cargo_bundle_version=$(cargo -q bundle --help 2>&1 | head -n 1 || echo "")
if [ "$cargo_bundle_version" != "cargo-bundle v0.6.0-zed" ]; then
cargo install cargo-bundle --git https://github.com/zed-industries/cargo-bundle.git --branch zed-deploy
fi

echo "Compiling zed binaries"
cargo build ${build_flag} --package ${zed_crate} --package cli

echo "Creating application bundle"
# TODO linux
# Here, hacks to make `cargo bundle` run work, but macOS does not need these
# Most probably, needs https://github.com/zed-industries/cargo-bundle/commit/9e185bd44d968d8039192220603494555afdbb4f from the upstream.
cp "target/${target_dir}/Zed" "target/${target_dir}/zed"
pushd crates/${zed_crate}
channel=$(<RELEASE_CHANNEL)
cp Cargo.toml Cargo.toml.backup
sed \
-i.backup -e \
"s/package.metadata.bundle-${channel}/package.metadata.bundle/" \
Cargo.toml

# TODO linux `zed_cli` does not get into this bundle despite being built
bundle_path=$(cargo bundle ${build_flag} --select-workspace-root | xargs)

mv Cargo.toml.backup Cargo.toml
popd

# For nightly, cut off the version out of the bundle name that `cargo bundle` always adds.
if [ "$channel" == "nightly" ]; then
version="$(cargo metadata --no-deps --manifest-path crates/zed/Cargo.toml --offline --format-version=1 | jq -r '.packages | map(select(.name == "zed"))[0].version')"
version_less_bundle_path=$(echo "$bundle_path" | sed "s/_$version//")
mv "$bundle_path" "$version_less_bundle_path"
bundle_path="$version_less_bundle_path"
fi

# TODO linux
# Other Linux systems will need a different set of manipulations + a way to know which ones to do.
# If bundle_name is not set or empty, use the basename of $bundle_path
if [ -z "${bundle_name}" ]; then
bundle_name=$(basename "${bundle_path}")
fi
# If bundle_name doesn't end in .deb, append it
if [[ "$bundle_name" != *.deb ]]; then
bundle_name="$bundle_name.deb"
fi

pushd target/
rm -rf bundle/ 2>/dev/null || true
dpkg-deb -x "${bundle_path}" bundle/
dpkg-deb --control "${bundle_path}" bundle/DEBIAN
mkdir -p bundle/usr/local/bin/
mv bundle/usr/bin/zed "bundle/usr/local/bin/zed-$channel"
cp "${target_dir}/cli" "bundle/usr/local/bin/cli-$channel"
ln -s "/usr/local/bin/cli-$channel" "bundle/usr/local/bin/zed"
rm -rf bundle/usr/bin/
dpkg-deb -b bundle/ "${target_dir}/${bundle_name}"
bundle_path="${PWD}/${target_dir}/${bundle_name}"
popd
echo "Bundled ${bundle_path}"
16 changes: 8 additions & 8 deletions script/bundle → script/bundle-mac
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/bin/bash
#!/usr/bin/env bash

set -e
set -euxo pipefail

build_flag="--release"
target_dir="release"
Expand All @@ -19,7 +19,7 @@ APPLE_NOTORIZATION_TEAM="MQ55VZLNZQ"
help_info() {
echo "
Usage: ${0##*/} [options] [bundle_name]
Build the application bundle.
Build the application bundle for macOS.
Options:
-d Compile in debug mode
Expand Down Expand Up @@ -81,8 +81,10 @@ done

shift $((OPTIND-1))

if [ "$1" ]; then
bundle_name=$1
if [[ $# -gt 0 ]]; then
if [ "$1" ]; then
bundle_name=$1
fi
fi

export ZED_BUNDLE=true
Expand All @@ -93,8 +95,6 @@ if [ "$cargo_bundle_version" != "cargo-bundle v0.6.0-zed" ]; then
cargo install cargo-bundle --git https://github.com/zed-industries/cargo-bundle.git --branch zed-deploy
fi

rustup target add wasm32-wasi

# Deal with versions of macOS that don't include libstdc++ headers
export CXXFLAGS="-stdlib=libc++"

Expand Down Expand Up @@ -208,7 +208,7 @@ else
echo "====== WARNING ======"

# NOTE: if you need to test universal links you have a few paths forward:
# - create a PR and tag it with the `run-build-dmg` label, and download the .dmg file from there.
# - create a PR and tag it with the `run-bundling` label, and download the .dmg file from there.
# - get a signing key for the MQ55VZLNZQ team from Nathan.
# - create your own signing key, and update references to MQ55VZLNZQ to your own team ID
# then comment out this line.
Expand Down
Loading

0 comments on commit ffebe2e

Please sign in to comment.