Merged
Conversation
jakule
requested review from
fheinecke,
r0mant and
tcsc
and removed request for
gabrielcorado and
smallinsky
r0mant
approved these changes
Mar 13, 2023
zmb3
approved these changes
Mar 16, 2023
jakule
force-pushed
the
jakule/increase-gha-build-timeout
branch
from
a0d2a7c to
a118484
Compare
Our ARM64 build fails quite frequently due to 30 minutes timeout. This PR increases the build timeout to one hour.
jakule
force-pushed
the
jakule/increase-gha-build-timeout
branch
from
a118484 to
68d98da
Compare
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
camscale
pushed a commit
that referenced
this pull request
Apr 12, 2023
Our ARM64 build fails quite frequently due to 30 minutes timeout. This PR increases the build timeout to one hour. Backport: #22926
fheinecke
pushed a commit
that referenced
this pull request
Apr 19, 2023
Our ARM64 build fails quite frequently due to 30 minutes timeout. This PR increases the build timeout to one hour.
camscale
pushed a commit
that referenced
this pull request
Apr 19, 2023
Our ARM64 build fails quite frequently due to 30 minutes timeout. This PR increases the build timeout to one hour. Backport: #22926
camscale
added a commit
that referenced
this pull request
Apr 19, 2023
* notarize: Accept parameters for dev and bundle ID Add a couple of parameters for the developer key ID and bundle ID for signing/notarizing binaries. Keep the hard-coded values as defaults for now, but we will remove these soon when all the call sites of the tool have been updated to pass these values. We want to parameterize these values so we can use different signing keys in GitHub Actions and to make the tool agnostic to which binaries it is signing. Backport: #23092 * release: Prepare for MacOS builds on GitHub Actions * release: Move Mac signing vars from script to Makefile Move the variables for Mac signing from the `build-common.sh` shell script to the `Makefile`. These vars will need to be passed to other build processes to parameterize the signing for different GitHub Actions build environments. The switch on `ENVIRONMENT_NAME` allows different secrets to be available in GitHub Actions for production (promote) vs developer (build) builds. The default environment name is `promote` so as to be compatible with the existing Drone setup, which does not define `ENVIRONMENT_NAME`. * release: Determine Mac signing key IDs automatically Remove the hard-coded MacOS signing key IDs from the Makefile and find them dynamically based on the name of the key. This allows GitHub Actions to be set up with new keys different to the ones on the Drone builders. As long as we keep the same name on the keys, we can rotate the keys without needing to update the IDs in the Makefile. This requires us to be more judicious about exporting the variables as exporting them causes them to be evaluated. We do not want to evaluate them on non-darwin targets, and on darwin, we should only evaluate it if needed for a recipe. So use a dynamic `eval` in the recipes that need the environment variables. * release: Pass key & team ID to notarize tool Override the hard-coded values in `notarize-apple-binaries` and pass the values we get based on the GitHub Actions environment. This allows us to sign and notarize software in a development branch more easily when working on the signing and notarizing process. This will not happen automatically, but it is expected that a developer can manually trigger a workflow to perform building, signing and notarizing from a dev branch where the workflow has temporarily changed the environment to `build`. A similar change to the `Makefile` in the teleport.e repository goes with this change. This adds a new bundle ID of `com.goteleport.dev` for the dev build of Teleport. This follows the same pattern as used for the dev build of the `tsh` binary and the current production bundle ID for Teleport. Previously there was no dev signing/notarizing process for the set of Teleport binaries. * release: Add script to setup the MacOS keychain for signing Add a script for setting up the MacOS keychain for signing applications and packages. It encapsulates the `security` commands to add either or both application keys and installer keys. The keys can be either base64-encoded in environment variables, or `.p12` files on disk, making it useful for local development. * release: Split MacOS signing vars into separate mk file Put the MacOS signing variables into a separate `.mk` file and include it from the main `Makefile`. Add more comments to document the purpose of the vars and where some of the values come from. * release: Add some more comments to keychain-setup.sh Explain that the purpose of the script is to be run on CI, but can also be run manually. Add the default values used to the usage message for the keychain and password. * Address PR comments on keychain-setup.sh script * Change shebang to /bin/bash * Use heredoc instead of multiple printfs for usage message * Move `local` declaration next to setting of kpath var * release: Export DEVELOPER_ID_APPLICATION in release-darwin The sub-make for enterprise needs this to be set or it cannot sign the enterprise binaries. Export it if we are doing signing/notarizing. Backport: #23407 * release: Dont look for signing keys if not needed Use a `make` conditional instead of a shell conditional when deciding whether to run notarization or not, based on APPLE_{USERNAME,PASSWORD} being set. When done as a shell conditional, $(DEVELOPER_ID_APPLICATION) was still being evaluated, and that causes a key error if the key does not exist in the keychain. If the `APPLE_{USERNAME,PASSWORD}` env vars are not set, it should not matter if the key is present or not as it will not be used. By switching to `$(if ...)` in make, the recipe will not be evaluated at all if the condition is false. To ensure the logs say what is going on, add a message when we do not notarize the binaries. Move the logic for notarization and checking the username/password into `darwin-signing.mk` with the other signing/notarization variables. This will make it reusable by the enterprise Makefile later. Backport: #23636 * release: Add build-connect target to Makefile * release: Add build-connect target to Makefile Add a `build-connect` target to the `Makefile` to build Teleport Connect via yarn on MacOS. Linux uses the `build.assets/Makefile` `teleterm` target, and Windows build pipelines do not use `make`. Add the `CSC_NAME` make variable containing the Developer Key ID to tell electron-builder which key to use to sign the package it produces. This gives us a little more control over how the Connect built and packaged and will simplify the CI scripts to have them just call make. It is also required in order to set the `CSC_NAME` environment name correctly as the developer ID is determined by the Makefile. * darwin: Fix echo command when not notarizing Remove the leading `@` from the echo command that is run when we are not notarizing binaries. It works fine from the OSS repo, but the enterprise repo needs to do `cd .. && $(NOTARIZE_COMMAND)` and the leading `@` causes the build error: `@echo: command not found`. We'll just have to live with a little extra noise amongst all the other noise. Backport: #23750 * release: Update build for product signing Update the build scripts to properly set up the key for signing packages using `productsign`, and parameterise the bundle ID for packages in the packaging scripts. Backport: #23820 * dronegen: Add workflow-tag arg to gh-trigger-workflow This was deliberately not backported in "[v12] Backport Distroless OCI builds (#22814)" (f1bfd5a) , but now we want it. * Increase GHA build timeout Our ARM64 build fails quite frequently due to 30 minutes timeout. This PR increases the build timeout to one hour. Backport: #22926 * drone: Switch Mac (darwin) pipelines to GitHub Actions * dronegen: Sort workflow inputs for stable output Sort the GitHub Actions inputs when generating the `gh-trigger-workflow` command line so that it does not randomly change order, as happens when iterating a map directly. * dronegen: Have darwin pipelines call out to GitHub Actions Update the darwin pipelines to run workflows on GitHub Actions instead of locally on drone builders. This replaces four pipelines with a single GitHub actions workflow as the one workflow builds the tarballs, Mac packages and Mac disk images. We continue to drive the push build from drone until we work out how secrets are safely managed in the Teleport OSS repo. * drone: Regenerate .drone.yml for Mac pipeline changes To regenerate the `.drone.yml` file, first three pipelines were manually removed: - build-darwin-amd64-pkg - build-darwin-amd64-pkg-tsh - build-darwin-amd64-connect Then make dronegen was run to update the pipelines: - push-build-darwin-amd64 - build-darwin-amd64 Backport: #24102 * Update `e` submodule for mac gha backport This commit will be updated after #1119 in teleport.e has been merged to point to the head of branch/v12. --------- Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com>
This was referenced Apr 19, 2023
fheinecke
added a commit
that referenced
this pull request
Apr 19, 2023
* Increase GHA build timeout (#22926) Our ARM64 build fails quite frequently due to 30 minutes timeout. This PR increases the build timeout to one hour. * Migrate publishing of new APT and YUM repos to GHA (#24350) * Deleted OPRT from this repo * Migrated dronegen to call gha workflow * Removed dead pipelines * Removed unused function to appease linter * Accounted for private repo promotions * Updated to use package name filter * Accounted for teleport-ent-updater publishing * Added missing `mkdir` call when determining if a release is a pre-release. (#24634) * Added missing mkdir call * Updated to calculate dirname at dronegen time * Removed "workflow-tag" argument from OPRT GHA call (#24637) * Flipped environment logic for OS package promotion (#24639) * Fixed OPRT migration issues (#24655) * Fixed OPRT migration issues * Removed package-to-test from unsupported teleport-ent-updater package * Added flag to `gh-trigger-workflow` tool to wait for pre-existing runs (#24696) * Added flag to `gh-trigger-workflow tool` to wait for pre-existing runs * Addressed PR comments * workflowId -> workflowID * Id -> ID * Updated promotion pipelines to publish OS packages in series (#24753) * Removed pipeline * Removed teleport-ent-updater deployment --------- Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com>
fheinecke
added a commit
that referenced
this pull request
Apr 19, 2023
* Increase GHA build timeout (#22926) Our ARM64 build fails quite frequently due to 30 minutes timeout. This PR increases the build timeout to one hour. * Migrate publishing of new APT and YUM repos to GHA (#24350) * Deleted OPRT from this repo * Migrated dronegen to call gha workflow * Removed dead pipelines * Removed unused function to appease linter * Accounted for private repo promotions * Updated to use package name filter * Accounted for teleport-ent-updater publishing * Added missing `mkdir` call when determining if a release is a pre-release. (#24634) * Added missing mkdir call * Updated to calculate dirname at dronegen time * Removed "workflow-tag" argument from OPRT GHA call (#24637) * Flipped environment logic for OS package promotion (#24639) * Fixed OPRT migration issues (#24655) * Fixed OPRT migration issues * Removed package-to-test from unsupported teleport-ent-updater package * Added flag to `gh-trigger-workflow` tool to wait for pre-existing runs (#24696) * Added flag to `gh-trigger-workflow tool` to wait for pre-existing runs * Addressed PR comments * workflowId -> workflowID * Id -> ID * Updated promotion pipelines to publish OS packages in series (#24753) * Removed pipeline * Removed teleport-ent-updater deployment --------- Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com>
fheinecke
added a commit
that referenced
this pull request
Apr 20, 2023
* Increase GHA build timeout (#22926) Our ARM64 build fails quite frequently due to 30 minutes timeout. This PR increases the build timeout to one hour. * Migrate publishing of new APT and YUM repos to GHA (#24350) * Deleted OPRT from this repo * Migrated dronegen to call gha workflow * Removed dead pipelines * Removed unused function to appease linter * Accounted for private repo promotions * Updated to use package name filter * Accounted for teleport-ent-updater publishing * Added missing `mkdir` call when determining if a release is a pre-release. (#24634) * Added missing mkdir call * Updated to calculate dirname at dronegen time * Removed "workflow-tag" argument from OPRT GHA call (#24637) * Flipped environment logic for OS package promotion (#24639) * Fixed OPRT migration issues (#24655) * Fixed OPRT migration issues * Removed package-to-test from unsupported teleport-ent-updater package * Added flag to `gh-trigger-workflow` tool to wait for pre-existing runs (#24696) * Added flag to `gh-trigger-workflow tool` to wait for pre-existing runs * Addressed PR comments * workflowId -> workflowID * Id -> ID * Updated promotion pipelines to publish OS packages in series (#24753) * Removed pipeline * Removed teleport-ent-updater deployment --------- Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com>
fheinecke
added a commit
that referenced
this pull request
Apr 20, 2023
* OS packaging and auto updates backport - v12 (#24781) * Increase GHA build timeout (#22926) Our ARM64 build fails quite frequently due to 30 minutes timeout. This PR increases the build timeout to one hour. * Migrate publishing of new APT and YUM repos to GHA (#24350) * Deleted OPRT from this repo * Migrated dronegen to call gha workflow * Removed dead pipelines * Removed unused function to appease linter * Accounted for private repo promotions * Updated to use package name filter * Accounted for teleport-ent-updater publishing * Added missing `mkdir` call when determining if a release is a pre-release. (#24634) * Added missing mkdir call * Updated to calculate dirname at dronegen time * Removed "workflow-tag" argument from OPRT GHA call (#24637) * Flipped environment logic for OS package promotion (#24639) * Fixed OPRT migration issues (#24655) * Fixed OPRT migration issues * Removed package-to-test from unsupported teleport-ent-updater package * Added flag to `gh-trigger-workflow` tool to wait for pre-existing runs (#24696) * Added flag to `gh-trigger-workflow tool` to wait for pre-existing runs * Addressed PR comments * workflowId -> workflowID * Id -> ID * Updated promotion pipelines to publish OS packages in series (#24753) * Removed pipeline * Removed teleport-ent-updater deployment --------- Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com> * Removed accidental backport of *-teleport-oci-distroless-images --------- Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com>
r0mant
pushed a commit
that referenced
this pull request
Apr 20, 2023
* OS packaging and auto updates backport - v12 (#24781) * Increase GHA build timeout (#22926) Our ARM64 build fails quite frequently due to 30 minutes timeout. This PR increases the build timeout to one hour. * Migrate publishing of new APT and YUM repos to GHA (#24350) * Deleted OPRT from this repo * Migrated dronegen to call gha workflow * Removed dead pipelines * Removed unused function to appease linter * Accounted for private repo promotions * Updated to use package name filter * Accounted for teleport-ent-updater publishing * Added missing `mkdir` call when determining if a release is a pre-release. (#24634) * Added missing mkdir call * Updated to calculate dirname at dronegen time * Removed "workflow-tag" argument from OPRT GHA call (#24637) * Flipped environment logic for OS package promotion (#24639) * Fixed OPRT migration issues (#24655) * Fixed OPRT migration issues * Removed package-to-test from unsupported teleport-ent-updater package * Added flag to `gh-trigger-workflow` tool to wait for pre-existing runs (#24696) * Added flag to `gh-trigger-workflow tool` to wait for pre-existing runs * Addressed PR comments * workflowId -> workflowID * Id -> ID * Updated promotion pipelines to publish OS packages in series (#24753) * Removed pipeline * Removed teleport-ent-updater deployment --------- Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com> * Resigned drone.yml --------- Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Our ARM64 build fails quite frequently due to 30 minutes timeout. This PR increases the build timeout to one hour.