Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Publish all artifacts for Linux ARM #4136

Closed
emilk opened this issue Nov 3, 2023 · 3 comments · Fixed by #5511
Closed

Publish all artifacts for Linux ARM #4136

emilk opened this issue Nov 3, 2023 · 3 comments · Fixed by #5511
Assignees
@abey79
Labels
enhancement New feature or request 🚢 CI
Milestone
0.15

Comments

@emilk
Copy link
Member

emilk commented Nov 3, 2023 •
edited
Loading

Consider BuildJet for CI, at least for Linux ARM

https://buildjet.com/for-github-actions

…at least for Linux:

We can offer both macOS M1 and Windows runners, but due to their respective licensing terms, we can't offer it as part of regular per minute offering. Therefore, we only offer runners per month, and we only take orders of 10 runners or more. For macOS M1 runners, we only offer persistent runners, but we can offer both ephemeral and persistent runners for Windows. If you're interested please reach out to us at [email protected], and we'll look into what we can do.

https://buildjet.com/for-github-actions/blog/launch-buildjet-cache

@jprochazk says:

the catch is they have significantly lower concurrency limits btw
you have to buy "concurrency extensions" (which cost extra outside of the usual per minute pricing). going beyond that seems like it may be more expensive than what we already have with gha
@emilk emilk added enhancement New feature or request 🚢 CI labels Nov 3, 2023
@emilk emilk added this to the Triage milestone Nov 3, 2023
@Wumpf Wumpf mentioned this issue Nov 3, 2023
Closed
@emilk emilk changed the title Try BuildJet for CI Consider BuildJet for CI Nov 6, 2023
@emilk
Copy link
Member Author

emilk commented Nov 7, 2023

Their caches are also small (20 GiB)

@emilk
Copy link
Member Author

emilk commented Nov 7, 2023

May still be worth it for Linux ARM, which GitHub doesn't have runners for

@emilk emilk changed the title Consider BuildJet for CI Consider BuildJet for CI, at least for Linux ARM Nov 7, 2023
@emilk emilk changed the title Consider BuildJet for CI, at least for Linux ARM Publish Python Wheels for Linux ARM Mar 12, 2024
@emilk emilk changed the title Publish Python Wheels for Linux ARM Publish all artifacts for Linux ARM Mar 12, 2024
@emilk
Copy link
Member Author

emilk commented Mar 12, 2024

From @teh-cmc :

GH arm64 runners have apparently entered private beta... https://github.blog/changelog/2023-10-30-accelerate-your-ci-cd-with-arm-based-hosted-runners-in-github-actions/

@Wumpf Wumpf mentioned this issue Mar 13, 2024
Merged
5 tasks
abey79 pushed a commit that referenced this issue Mar 14, 2024
@Wumpf
Build C++ SDK for Linux ARM64 (#5489)
1030934 
### What

* Part of #4136

Had to use a buildjet runner because some of our `rerun_c` dependencies
(`ring` foremost) had trouble cross compiling after all.

Artifact I built from CI by emulating push-to-main ci:
https://build.rerun.io/commit/04b1eb6/rerun_cpp_sdk.zip

Not properly tested, but evidently has arm64 linux static library in it.


### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested the web demo (if applicable):
* Using newly built examples:
[app.rerun.io](https://app.rerun.io/pr/5489/index.html)
* Using examples from latest `main` build:
[app.rerun.io](https://app.rerun.io/pr/5489/index.html?manifest_url=https://app.rerun.io/version/main/examples_manifest.json)
* Using full set of examples from `nightly` build:
[app.rerun.io](https://app.rerun.io/pr/5489/index.html?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG
* [x] If applicable, add a new check to the [release
checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)!

- [PR Build Summary](https://build.rerun.io/pr/5489)
- [Docs
preview](https://rerun.io/preview/20c26b4682d0d874dddbaab26102b8a061a885a3/docs)
<!--DOCS-PREVIEW-->
- [Examples
preview](https://rerun.io/preview/20c26b4682d0d874dddbaab26102b8a061a885a3/examples)
<!--EXAMPLES-PREVIEW-->
- [Recent benchmark results](https://build.rerun.io/graphs/crates.html)
- [Wasm size tracking](https://build.rerun.io/graphs/sizes.html)
This was referenced Mar 14, 2024
Merged
Closed
Wumpf added a commit that referenced this issue Mar 14, 2024
@abey79 @Wumpf
Build CLI for Linux ARM64 (#5503)
56bb2ea 
### What

Add support for building a Linux ARM64 version of the CLI in our build
system.

- Follow-up to #5489
- Part of #4136

This PR introduces ugly work-around to be cleaned when our pixi deps
support linux-aarch64:
- #5507

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested the web demo (if applicable):
* Using newly built examples:
[app.rerun.io](https://app.rerun.io/pr/5503/index.html)
* Using examples from latest `main` build:
[app.rerun.io](https://app.rerun.io/pr/5503/index.html?manifest_url=https://app.rerun.io/version/main/examples_manifest.json)
* Using full set of examples from `nightly` build:
[app.rerun.io](https://app.rerun.io/pr/5503/index.html?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG
* [x] If applicable, add a new check to the [release
checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)!

- [PR Build Summary](https://build.rerun.io/pr/5503)
- [Docs
preview](https://rerun.io/preview/65c8082ad0c5482bad9167e8620490494f80637e/docs)
<!--DOCS-PREVIEW-->
- [Examples
preview](https://rerun.io/preview/65c8082ad0c5482bad9167e8620490494f80637e/examples)
<!--EXAMPLES-PREVIEW-->
- [Recent benchmark results](https://build.rerun.io/graphs/crates.html)
- [Wasm size tracking](https://build.rerun.io/graphs/sizes.html)

---------

Co-authored-by: Andreas Reich <[email protected]>
@abey79 abey79 mentioned this issue Mar 14, 2024
Merged
5 tasks
abey79 added a commit that referenced this issue Mar 18, 2024
@abey79 @jleibs
Build wheels for Linux ARM64 (#5511)
be1fbcc 
### What

Build linux-aarch64 wheels (with manylinux_2_31) and fixes the C and CLI
builds to bring the min required glibc version to 2.31.

**Note**: cursory inspection indicates that the _actual_ min glibc
version 2.29. However, we don't formally test that. The only test is
done by `maturin` on our wheels, against the `manylinux_2_31` standard.

Amongst other things, this PR:
- updates our ci docker image to be compatible with aarch64 (#5543 –
thanks @jleibs)
- remove all instances of double GCS authentication (our setup-rust
action auth to GCS, so an explicit auth isn't necessary, _and_ would
break cleanup in linux-arm/cli build workflow—why there only is unclear)
- use it for all linux-aarch64 builds
- fixes `pixi.toml` for linux-aarch64 compatibility
- bumps pixi to 0.16.1 everywhere in the ci
- adds a manual trigger for building wheels
- (mostly) unifies the target naming scheme

#### Related

- Follow-up to:
  - #5489 
  - #5503 
- Closes #4136 
- Fixes #5507

#### Further work
- ~~support lower glibc version:
#5512
- test more wheels: #5525

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested the web demo (if applicable):
* Using newly built examples:
[app.rerun.io](https://app.rerun.io/pr/5511/index.html)
* Using examples from latest `main` build:
[app.rerun.io](https://app.rerun.io/pr/5511/index.html?manifest_url=https://app.rerun.io/version/main/examples_manifest.json)
* Using full set of examples from `nightly` build:
[app.rerun.io](https://app.rerun.io/pr/5511/index.html?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG
* [x] If applicable, add a new check to the [release
checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)!

- [PR Build Summary](https://build.rerun.io/pr/5511)
- [Docs
preview](https://rerun.io/preview/c029b411730d035b9babdba3f9721dbe2905196b/docs)
<!--DOCS-PREVIEW-->
- [Examples
preview](https://rerun.io/preview/c029b411730d035b9babdba3f9721dbe2905196b/examples)
<!--EXAMPLES-PREVIEW-->
- [Recent benchmark results](https://build.rerun.io/graphs/crates.html)
- [Wasm size tracking](https://build.rerun.io/graphs/sizes.html)

---------

Co-authored-by: Jeremy Leibs <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@abey79 abey79

Labels
enhancement New feature or request 🚢 CI
Projects
None yet
Milestone
0.15
Development

Successfully merging a pull request may close this issue.

Build wheels for Linux ARM64 rerun-io/rerun
3 participants
@emilk @nikolausWest @abey79