Add googleStorageUploadExt step by v1v · Pull Request #867 · elastic/apm-pipeline-library

v1v · 2020-12-01T16:47:28Z

What does this PR do?

Support step to interact with the google storage

Why is it important?

Workaround jenkinsci/google-storage-plugin#61

Tests

Took 16 seconds to install gsutil and run the copy to the storage, see stage that uploaded the artifacts to google storage here

Beats

build

ghost · 2020-12-01T16:54:31Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Pull request #867 updated
Start Time: 2021-02-16T09:30:49.601+0000
Duration: 6 min 47 sec
Commit: 2756a39

Test stats 🧪

Test	Results
Failed	0
Passed	1046
Skipped	2
Total	1048

Trends 🧪

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	1046
Skipped	2
Total	1048

…google-storage-step * upstream/master: (111 commits) Image build pipeline for flaky v2 (#968) fix: enable sandbox for pipelines (#967) test: taest jobDSL job when them changed (#966) docs: update CHANGELOG.md [maven-release-plugin] prepare for next development iteration [maven-release-plugin] prepare release v1.1.185 feat: Archive only on fail (#952) Disable getStageId call (#962) docs: side note to setEnvVar step (#957) Add mergify bot (#958) Mergify: configuration update (#891) docs: update CHANGELOG.md [maven-release-plugin] prepare for next development iteration [maven-release-plugin] prepare release v1.1.184 fix: resize the partition size to 100GB to use all the space (#953) fix: GO_VERSION can be undefined (#954) chore: bump Go version to 1.15.7 (#951) [docs][matrix] explain the new step (#948) Show build and test trends as PR comments and GitHub .Status check (#942) feat: custom matrix step (#938) ...

… feature/override-google-storage-step * 'master' of github.com:elastic/apm-pipeline-library: [chore]: delegate delete to the core (#904) docs: update CHANGELOG.md [maven-release-plugin] prepare for next development iteration [maven-release-plugin] prepare release v1.1.186 chore: refactor jobs in folders (#974) Fix broken documentation (#970) Add releaseNotification step to simplify agent release pipelines (#976) chore(deps-dev): bump junit from 4.13.1 to 4.13.2 (#980) Disable/Enable pre-commit skipped tests in the report (#977) feat: support parameters for githubPrCheckApproved (#910) Automatically merge PRs for the 7.x branch (#971) chore: refactor params variables to args (#960)

v1v · 2021-02-16T09:05:57Z

elastic/beats#24048 is the consumer :) and it works like a charm

v1v · 2021-02-16T09:06:45Z

+  def url = 'https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-319.0.0'
+  url = "${url}-linux-${is64() ? 'x86_64' : 'x86'}.tar.gz"


This is not ARM supported but only linux 64/32 bits

But it works anyway

See build

Co-authored-by: cachedout <mike.place@elastic.co>

* Add googleStorageUploadExt step * chore: for testing purposes * Fix wrong argument * Add UTs * Leftovers for the copy * Minor refactor to use script * Add ITs * Add strip * Use the right credentials id * Use the temp bucket * Use the protocol * Add JCasC for the Google Storage credentials * Use another target * Debugging purposes * Use another approach * Use top level options to run in parallel and quietly * Create stage only for the google storage step * Warn only since it will fail when executed and use the master branch * Update .ci/Jenkinsfile * Apply suggestions from code review Co-authored-by: cachedout <mike.place@elastic.co> * Only in master for the time being Co-authored-by: cachedout <mike.place@elastic.co> (cherry picked from commit 4bf7ff4)

* Add googleStorageUploadExt step * chore: for testing purposes * Fix wrong argument * Add UTs * Leftovers for the copy * Minor refactor to use script * Add ITs * Add strip * Use the right credentials id * Use the temp bucket * Use the protocol * Add JCasC for the Google Storage credentials * Use another target * Debugging purposes * Use another approach * Use top level options to run in parallel and quietly * Create stage only for the google storage step * Warn only since it will fail when executed and use the master branch * Update .ci/Jenkinsfile * Apply suggestions from code review Co-authored-by: cachedout <mike.place@elastic.co> * Only in master for the time being Co-authored-by: cachedout <mike.place@elastic.co> (cherry picked from commit 4bf7ff4) Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* upstream/7.x: Fix the getCause and getCauses for the AbortException (#1041) (#1042) gsutil for windows with python (#1008) (#1009) Report only one DeleteDir step failure (#1006) (#1007) [cosmetic] Replace multiline chars in the GitHub PR comment (#998) (#1003) [cosmetic] add the author (#999) (#1002) chore: use opentelemetry release (#1000) (#1001) Avoid cache gsutils temporary location (#994) (#995) [JCasC] google cloud engine local setup (#979) (#992) Support curl in addition to wget for gsutil (#988) (#989) Support windows for gsutil (#985) (#987) Add googleStorageUploadExt step (#867) (#982) Add releaseNotification step to simplify agent release pipelines (#976) (#981)

v1v added 2 commits December 1, 2020 16:40

Add googleStorageUploadExt step

9fcadbd

chore: for testing purposes

4d8b50a

v1v and others added 19 commits December 1, 2020 17:01

Fix wrong argument

6ceaf6c

Add UTs

a7afa85

Leftovers for the copy

1f23f03

Minor refactor to use script

29672fe

Add ITs

57133e9

Add strip

0be9457

Merge branch 'master' into feature/override-google-storage-step

5ddce47

Use the right credentials id

d546d4b

Use the temp bucket

9de1003

Use the protocol

9bec009

Add JCasC for the Google Storage credentials

5eda46e

Use another target

b455342

Debugging purposes

b475520

Use another approach

9a21048

Use top level options to run in parallel and quietly

f748ad4

Create stage only for the google storage step

4d5f2de

Warn only since it will fail when executed and use the master branch

6d7f7fc

v1v commented Feb 15, 2021

View reviewed changes

Comment thread .ci/Jenkinsfile Outdated

v1v marked this pull request as ready for review February 15, 2021 18:35

v1v requested a review from a team February 15, 2021 18:35

v1v self-assigned this Feb 15, 2021

v1v added automation ci labels Feb 15, 2021

botelastic Bot added groovy master labels Feb 15, 2021

Update .ci/Jenkinsfile

1431a71

v1v mentioned this pull request Feb 15, 2021

[CI] googleStorageUploadExt step elastic/beats#24048

Merged

v1v commented Feb 16, 2021

View reviewed changes

kuisathaverat approved these changes Feb 16, 2021

View reviewed changes

cachedout approved these changes Feb 16, 2021

View reviewed changes

Comment thread docs/STORAGE.md Outdated

Comment thread docs/STORAGE.md Outdated

Apply suggestions from code review

8454005

Co-authored-by: cachedout <mike.place@elastic.co>

v1v added backport-to-7.x Automatically backport PRs from master ready-to-merge labels Feb 16, 2021

Only in master for the time being

2756a39

v1v added ready-to-merge and removed ready-to-merge labels Feb 16, 2021

mergify Bot merged commit 4bf7ff4 into elastic:master Feb 16, 2021

mergify Bot mentioned this pull request Feb 16, 2021

Add googleStorageUploadExt step (bp #867) #982

Merged

v1v deleted the feature/override-google-storage-step branch February 16, 2021 10:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add googleStorageUploadExt step#867

Add googleStorageUploadExt step#867
mergify[bot] merged 24 commits intoelastic:masterfrom
v1v:feature/override-google-storage-step

v1v commented Dec 1, 2020 •

edited

Loading

Uh oh!

ghost commented Dec 1, 2020 •

edited by ghost

Loading

Build stats

Test stats 🧪

Trends 🧪

Test stats 🧪

Uh oh!

Uh oh!

v1v commented Feb 16, 2021

Uh oh!

v1v Feb 16, 2021

Uh oh!

v1v Feb 16, 2021

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		def url = 'https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-319.0.0'
		url = "${url}-linux-${is64() ? 'x86_64' : 'x86'}.tar.gz"

Conversation

v1v commented Dec 1, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Why is it important?

Tests

Beats

Uh oh!

ghost commented Dec 1, 2020 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

Trends 🧪

💚 Flaky test report

Test stats 🧪

Uh oh!

Uh oh!

v1v commented Feb 16, 2021

Uh oh!

v1v Feb 16, 2021

Choose a reason for hiding this comment

Uh oh!

v1v Feb 16, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

v1v commented Dec 1, 2020 •

edited

Loading

ghost commented Dec 1, 2020 •

edited by ghost

Loading