Setup binary release CI pipeline#1085
Merged
mergify[bot] merged 11 commits intomasterfrom Jul 2, 2020
Merged
Conversation
4e266b2 to
bd731c0
Compare
bowd
added a commit
that referenced
this pull request
Jul 1, 2020
Required by #1085 to enable compiling celo-blockchain for windows
nategraf
approved these changes
Jul 2, 2020
nategraf
left a comment
There was a problem hiding this comment.
Looks good! Just a few out-of-date comments
c930f60 to
69b1e0b
Compare
69b1e0b to
a4142e6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR includes the necessary bits to setup a CI pipeline that cross-compiles all geth tools into release binaries for the platforms we want/support. Currently these are:
pending on Update gosigar usage #1082merged!)pending on celo-bls-go v0.1.6 releasepending on Bump celo-bls-go to v0.1.6 #1089merged!)Elements of the PR:
Dockerfile.binariesfor the container where cross-compilation occurs. It's based on a fork of xgo, and includes some back-ported mingw packages for the windows build.Changes to the
gethbuild scripts:internal/build/env.gowhich which can be used for Google Cloud Build.build/ci.gothat bundles the binaries into well named release archivecloudbuild-binaries.yamlwhich defines the CI pipeline. It should be used in conjunction with a trigger onrelease/.*andmasterbranches and the trigger should have to variables:_BUCKETwith the name of the google cloud storage bucket where the artefacts will be stored_BUILD_TARGETScomma-separated string of build targets. e.g.linux/amd64,linux/386Shortcomings
build.Environmentstruct requires the commit timestamp, which is then passed tomain.gitDatein the binaries that it builds. Usually this is extracted from the git but in Cloud Build in the CI steps the git data is stripped and all we have is what's passed through env vars. I have substituted the commit timestamp with the build timestamp instead.xgoimage is really large (3gb) and adds about ~4minutes to the build timeRelease artefacts
The pipeline outputs release artefacts into
<bucket>/<branch>/<file>. For example here's the output of a test build:A few things to notice here:
The folder is
<bucket>/release/1.1yet the version is1.0.0-unstablethis is because there isn't any enforced relationship between the branch name and the version sourced fromparams/version.gowhich is the authoritative source. So in this case I just created the dummyrelease/1.1branch on my fork in order to play around. Because we're using the branch name in the path all "nightly" version will be stored in the<bucket>/masterfolder, if we setup the trigger onmaster.There are 2 archives per platform, one containing only⚠️ The archives also contain the "COPYING" file, we need to decide if we want to make changes to that ⚠️
gethand the other containing all binary tools located insidecmd, just like geth releases.Post-merge TODOs:
celo-testnetto start running the pipelineTested
I have currently tested the CI pipeline in a personal google cloud project and tested the linux binaries in docker.
Related issues
Backwards compatibility
Changes are only related to tooling so no problem with compatibility.