Conversation
mergify Bot
pushed a commit
that referenced
this pull request
Jul 2, 2020
### Description This PR includes the necessary bits to setup a CI pipeline that cross-compiles all geth tools into release binaries for the platforms we want/support. Currently these are: - ✅Linux 32bit/64bit - ✅Darwin 32bit/64bit (~pending on #1082~ merged!) - ✅Windows 64bit (~pending on celo-bls-go v0.1.6 release~ ~pending on #1089~ merged!) Elements of the PR: - `Dockerfile.binaries` for the container where cross-compilation occurs. It's based on a [fork of xgo](https://github.com/techknowlogick/xgo), and includes some back-ported mingw packages for the windows build. - Changes to the `geth` build scripts: - Introduce a new CI environment in `internal/build/env.go` which which can be used for Google Cloud Build. - Create a new ci task in `build/ci.go` that bundles the binaries into well named release archive - `cloudbuild-binaries.yaml` which defines the CI pipeline. It should be used in conjunction with a trigger on `release/.*` and `master` branches and the trigger should have to variables: - `_BUCKET` with the name of the google cloud storage bucket where the artefacts will be stored - `_BUILD_TARGETS` comma-separated string of build targets. e.g. `linux/amd64,linux/386` #### Shortcomings - 🔴 The geth `build.Environment` struct requires the commit timestamp, which is then passed to `main.gitDate` in the binaries that it builds. Usually this is extracted from the git but in Cloud Build in the CI steps the git data is stripped and all we have is what's passed through env vars. I have substituted the commit timestamp with the build timestamp instead. - 🟡 The `xgo` image is really large (3gb) and adds about ~4minutes to the build time #### Release artefacts The pipeline outputs release artefacts into `<bucket>/<branch>/<file>`. For example here's the output of a test build: <img width="553" alt="Screenshot 2020-06-29 at 11 41 00" src="https://user-images.githubusercontent.com/304771/85994959-1433eb00-b9fe-11ea-9180-22ea95cb774c.png"> A few things to notice here: - The folder is `<bucket>/release/1.1` yet the version is `1.0.0-unstable` this is because there isn't any enforced relationship between the branch name and the version sourced from `params/version.go` which is the authoritative source. So in this case I just created the dummy `release/1.1` branch on my fork in order to play around. Because we're using the branch name in the path all "nightly" version will be stored in the `<bucket>/master` folder, if we setup the trigger on `master`. - There are 2 archives per platform, one containing only `geth` and the other containing all binary tools located inside `cmd`, just like geth releases.⚠️ The archives also contain the "COPYING" file, we need to decide if we want to make changes to that⚠️ #### Post-merge TODOs: - [x] Create a bucket and triggers in `celo-testnet` to start running the pipeline - [x] Enable more build targets as they are unblocked ### Tested I have currently tested the CI pipeline in a personal google cloud project and tested the linux binaries in docker. ### Related issues - Fixes #1073 ### Backwards compatibility Changes are only related to tooling so no problem with compatibility.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Bumps the
celo-bls-godependency tov0.1.6which fixes a windows linking issue and allowscelo-blockchainto compile and run on windows.Other changes
I also ran a
go mod tidyto clean up dependencies, that's the cause of most of the modifications. In my experience it works well but people don't feel comfortable running it because it introduces unrelated changes but the reverse of the coin is that a lot of noise builds up in the deps.Tested
I have compiled
celo-blockchainfor windows with the bumped dependency and I'm syncing a node in a VM.Related issues
Backwards compatibility
The change in
celo-bls-goonly affects the library on windows which wasn't linking at all before this change so it doesn't affect linux/darwin builds.