Populate errorlevel from the top level Makefile by v1v · Pull Request #69 · elastic/golang-crossbuild

v1v · 2020-11-10T09:22:48Z

What

Populate the error level in the foreach, for such I use a file flag with the errorlevel since I could not find the way to use dynamic variables within the make target.

Why

As far as I see the CI uses the toplevel Makefile:

build
push

Those goals call the other makefiles which already contain the exit 1 if something bad happened (

golang-crossbuild/go1.14/Makefile.debian7

Line 8 in 7fe9c7c

@$(foreach var,$(IMAGES),$(MAKE) -C $(var) build || exit 1;)

) but it seems they are not populated as expected

For instance:

curl -s 'https://beats-ci.elastic.co/job/Beats/job/golang-crossbuild-mbp/view/change-requests/job/PR-68/1/consoleText'  | grep failed                                                                                        
[2020-11-09T15:26:26.723Z] ../Makefile.common:13: recipe for target 'build' failed
[2020-11-09T15:26:26.723Z] Makefile:4: recipe for target 'build' failed
[2020-11-09T15:36:41.288Z] ../Makefile.common:13: recipe for target 'build' failed
[2020-11-09T15:36:41.288Z] Makefile:4: recipe for target 'build' failed
[2020-11-09T15:44:57.671Z] ../Makefile.common:13: recipe for target 'build' failed
[2020-11-09T15:44:57.671Z] Makefile:4: recipe for target 'build' failed
[2020-11-09T15:53:33.596Z] ../Makefile.common:13: recipe for target 'build' failed
[2020-11-09T15:53:33.596Z] Makefile:4: recipe for target 'build' failed
[2020-11-09T15:53:47.863Z] ../Makefile.common:13: recipe for target 'build' failed
[2020-11-09T15:53:47.863Z] Makefile:4: recipe for target 'build' failed
[2020-11-09T15:54:01.816Z] ../Makefile.common:13: recipe for target 'build' failed
[2020-11-09T15:54:01.816Z] Makefile:4: recipe for target 'build' failed
[2020-11-09T16:30:17.820Z] ..//../Makefile.common:9: recipe for target 'push' failed
[2020-11-09T16:30:17.820Z] Makefile:8: recipe for target 'push' failed
[2020-11-09T16:38:05.519Z] ..//../Makefile.common:9: recipe for target 'push' failed
[2020-11-09T16:38:05.519Z] Makefile:8: recipe for target 'push' failed
[2020-11-09T16:45:36.771Z] ..//../Makefile.common:9: recipe for target 'push' failed
[2020-11-09T16:45:36.771Z] Makefile:8: recipe for target 'push' failed

But the build was successful

Tests

Prepare context, so remove the number of go targets to just one -> sed -i.back 's#TARGETS=.*#TARGETS=go1.10#g' Makefile
Change dockerfile to fail earlier sed -i.back 's#apt-get#ap-get#g' go1.10/base/Dockerfile.tmpl
Then try with make build and the output should look like:

>> Building docker.elastic.co/beats-dev/golang-crossbuild:1.10.8-base
Sending build context to Docker daemon  13.31kB
Step 1/18 : ARG DEBIAN_VERSION
Step 2/18 : FROM debian:${DEBIAN_VERSION}
 ---> c4ccba324c9c
Step 3/18 : RUN     apt-et -o Acquire::Check-Valid-Until=false update         && apt-get dist-upgrade -y         && apt-get install -y --no-install-recommends --allow-unauthenticated             build-essential             ca-certificates             curl             git             gnupg             make             file             flex             bison         && rm -rf /var/lib/apt/lists/*
 ---> Running in 587d7e10bf12
/bin/sh: 1: apt-et: not found
The command '/bin/sh -c apt-et -o Acquire::Check-Valid-Until=false update         && apt-get dist-upgrade -y         && apt-get install -y --no-install-recommends --allow-unauthenticated             build-essential             ca-certificates             curl             git             gnupg             make             file             flex             bison         && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 127
make[2]: *** [build] Error 127
make[1]: *** [build] Error 1
>> Building docker.elastic.co/beats-dev/golang-crossbuild:1.10.8-base-debian7
Sending build context to Docker daemon  13.31kB
Step 1/19 : ARG DEBIAN_VERSION
Step 2/19 : FROM debian:${DEBIAN_VERSION}
 ---> 10fcec6d95c4
Step 3/19 : COPY sources-debian7.list /etc/apt/sources.list
 ---> Using cache
 ---> 6eb5bc1695ac
Step 4/19 : RUN     apt-et -o Acquire::Check-Valid-Until=false update         && apt-get dist-upgrade -y         && apt-get install -y --no-install-recommends --allow-unauthenticated             build-essential             ca-certificates             curl             git             gnupg             make             file             flex             bison         && rm -rf /var/lib/apt/lists/*
 ---> Running in 2a543e4e969f
/bin/sh: 1: apt-et: not found
The command '/bin/sh -c apt-et -o Acquire::Check-Valid-Until=false update         && apt-get dist-upgrade -y         && apt-get install -y --no-install-recommends --allow-unauthenticated             build-essential             ca-certificates             curl             git             gnupg             make             file             flex             bison         && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 127
make[2]: *** [build] Error 127
make[1]: *** [build] Error 1
>> Building docker.elastic.co/beats-dev/golang-crossbuild:1.10.8-base-debian8
Sending build context to Docker daemon  13.31kB
Step 1/19 : ARG DEBIAN_VERSION
Step 2/19 : FROM debian:${DEBIAN_VERSION}
 ---> 091099bf65ad
Step 3/19 : COPY sources-debian8.list /etc/apt/sources.list
 ---> Using cache
 ---> d710270801fe
Step 4/19 : RUN     apt-et -o Acquire::Check-Valid-Until=false update         && apt-get dist-upgrade -y         && apt-get install -y --no-install-recommends --allow-unauthenticated             build-essential             ca-certificates             curl             git             gnupg             make             file             flex             bison         && rm -rf /var/lib/apt/lists/*
 ---> Running in 79dbf5c58532
/bin/sh: 1: apt-et: not found
The command '/bin/sh -c apt-et -o Acquire::Check-Valid-Until=false update         && apt-get dist-upgrade -y         && apt-get install -y --no-install-recommends --allow-unauthenticated             build-essential             ca-certificates             curl             git             gnupg             make             file             flex             bison         && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 127
make[2]: *** [build] Error 127
make[1]: *** [build] Error 1
>> Building docker.elastic.co/beats-dev/golang-crossbuild:1.10.8-base-debian9
Sending build context to Docker daemon  13.31kB
Step 1/18 : ARG DEBIAN_VERSION
Step 2/18 : FROM debian:${DEBIAN_VERSION}
 ---> c4ccba324c9c
Step 3/18 : RUN     apt-et -o Acquire::Check-Valid-Until=false update         && apt-get dist-upgrade -y         && apt-get install -y --no-install-recommends --allow-unauthenticated             build-essential             ca-certificates             curl             git             gnupg             make             file             flex             bison         && rm -rf /var/lib/apt/lists/*
 ---> Running in e0ce27ea9e43
/bin/sh: 1: apt-et: not found
The command '/bin/sh -c apt-et -o Acquire::Check-Valid-Until=false update         && apt-get dist-upgrade -y         && apt-get install -y --no-install-recommends --allow-unauthenticated             build-essential             ca-certificates             curl             git             gnupg             make             file             flex             bison         && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 127
make[2]: *** [build] Error 127
make[1]: *** [build] Error 1
>> Building docker.elastic.co/beats-dev/fpm:1.11.0
Sending build context to Docker daemon   7.68kB
Step 1/16 : FROM debian:stretch
 ---> c4ccba324c9c
Step 2/16 : LABEL maintainer="Elastic Beats Team"
 ---> Using cache
 ---> b2aef5f6da79
Step 3/16 : RUN     apt-get update &&     apt-get install -y --no-install-recommends         autoconf build-essential libffi-dev ruby-dev rpm zip dos2unix libgmp3-dev         curl make gcc     && rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> 15a993c744df
Step 4/16 : ARG SUEXEC_VERSION=0.2
 ---> Using cache
 ---> 40880406f645
Step 5/16 : ARG SUEXEC_DOWNLOAD_URL=https://github.com/ncopa/su-exec/archive/v${SUEXEC_VERSION}.tar.gz
 ---> Using cache
 ---> dae6a849d3ed
Step 6/16 : ARG SUEXEC_DOWNLOAD_SHA256=ec4acbd8cde6ceeb2be67eda1f46c709758af6db35cacbcde41baac349855e25
 ---> Using cache
 ---> 418019c00a04
Step 7/16 : RUN     curl -fsSL "$SUEXEC_DOWNLOAD_URL" -o suexec.tar.gz     && echo "$SUEXEC_DOWNLOAD_SHA256  suexec.tar.gz" | sha256sum -c -     && mkdir /suexec     && tar -C /suexec --strip-components=1 -xzf suexec.tar.gz     && make -C /suexec     && mv /suexec/su-exec /usr/bin/     && rm -rf /suexec suexec.tar.gz
 ---> Using cache
 ---> 0f09f49578e2
Step 8/16 : ARG FPM_VERSION
 ---> Using cache
 ---> 68c9011dc3ce
Step 9/16 : RUN gem install fpm -v "$FPM_VERSION"
 ---> Using cache
 ---> 0ac98da3b18f
Step 10/16 : COPY rootfs /
 ---> Using cache
 ---> 51a14c941403
Step 11/16 : ENTRYPOINT  ["/entrypoint.sh"]
 ---> Using cache
 ---> bd78da502078
Step 12/16 : ARG BUILD_DATE
 ---> Using cache
 ---> 6a1db4aa6593
Step 13/16 : ARG IMAGE
 ---> Using cache
 ---> 232d152194ef
Step 14/16 : ARG VCS_REF
 ---> Using cache
 ---> 83f078880256
Step 15/16 : ARG VCS_URL
 ---> Using cache
 ---> b2d5a1425a20
Step 16/16 : LABEL org.label-schema.build-date=$BUILD_DATE       org.label-schema.name=$IMAGE       org.label-schema.vcs-ref=$VCS_REF       org.label-schema.vcs-url=$VCS_URL       org.label-schema.schema-version="1.0"
 ---> Running in 10644c603a37
Removing intermediate container 10644c603a37
 ---> 944b37cd8d4b
Successfully built 944b37cd8d4b
Successfully tagged docker.elastic.co/beats-dev/fpm:1.11.0
exit $(cat ".status.build")
make: *** [build] Error 1

elasticmachine · 2020-11-10T10:15:20Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: [Pull request #69 updated]
Start Time: 2020-11-10T10:06:22.666+0000
Duration: 66 min 19 sec

v1v · 2020-11-10T12:09:51Z

Failed for some broken builds

$ curl -s https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/pipelines/golang-crossbuild-mbp/branches/PR-69/runs/4/nodes/75/steps/89/log/\?start\=0 | grep failed
[2020-11-10T10:41:23.201Z] ../Makefile.common:13: recipe for target 'build' failed
[2020-11-10T10:41:23.201Z] Makefile.debian7:8: recipe for target 'build' failed
[2020-11-10T10:48:16.989Z] ../Makefile.common:13: recipe for target 'build' failed
[2020-11-10T10:48:16.989Z] Makefile:4: recipe for target 'build' failed
[2020-11-10T10:56:20.614Z] ../Makefile.common:13: recipe for target 'build' failed
[2020-11-10T10:56:20.614Z] Makefile:4: recipe for target 'build' failed
[2020-11-10T11:00:18.884Z] Makefile:5: recipe for target 'build' failed

kuisathaverat · 2020-11-10T14:37:23Z

@@ -1,20 +1,26 @@
 TARGETS=go1.10 go1.11 go1.12 go1.13 go1.14

+build: status=".status.build"


I think you only need .SHELLFLAGS = -ec this will stop make on the first failure

.ONESHELL : also can work

I think fail fast might not be ideal for this particular use case, since the build takes a while, I'd prefer to run and then evaluate the outcome to provide as many details as possible, what do you think?

you know what happens when you allow keeping things broken 😄

error should be reported and fail the build always but at the end, so that's what I meant

v1v · 2020-11-11T09:16:08Z

I'll merge this PR since it does nothing but validating the make calls did not fail and if so failed the build at the end.

Populate errorlevel from the top level Makefile

2fc8b65

v1v mentioned this pull request Nov 10, 2020

Add support for Golang 1.15.4 #68

Closed

andresrc requested review from andrewkroh, kvch and urso and removed request for urso November 10, 2020 09:34

andresrc added the Team:Automation Label for the Observability productivity team label Nov 10, 2020

kvch approved these changes Nov 10, 2020

View reviewed changes

Use flag file to report the errorlevel

1eee7a2

v1v requested a review from a team November 10, 2020 10:10

v1v self-assigned this Nov 10, 2020

v1v added the bug Something isn't working label Nov 10, 2020

chore: remove extra spaces

e4acbd5

v1v force-pushed the feature/fail-if-error branch from e8862ff to e4acbd5 Compare November 10, 2020 10:12

v1v marked this pull request as ready for review November 10, 2020 10:12

mdelapenya approved these changes Nov 10, 2020

View reviewed changes

kuisathaverat reviewed Nov 10, 2020

View reviewed changes

kuisathaverat approved these changes Nov 10, 2020

View reviewed changes

v1v merged commit a61d5e1 into elastic:master Nov 11, 2020

v1v deleted the feature/fail-if-error branch November 11, 2020 09:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Populate errorlevel from the top level Makefile#69

Populate errorlevel from the top level Makefile#69
v1v merged 3 commits into
elastic:masterfrom
v1v:feature/fail-if-error

v1v commented Nov 10, 2020 •

edited

Loading

Uh oh!

elasticmachine commented Nov 10, 2020 •

edited

Loading

Build stats

Uh oh!

v1v commented Nov 10, 2020

Uh oh!

kuisathaverat Nov 10, 2020

Uh oh!

kuisathaverat Nov 10, 2020

Uh oh!

v1v Nov 10, 2020 •

edited

Loading

Uh oh!

kuisathaverat Nov 10, 2020

Uh oh!

v1v Nov 10, 2020

Uh oh!

v1v commented Nov 11, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

		@@ -1,20 +1,26 @@
		TARGETS=go1.10 go1.11 go1.12 go1.13 go1.14

		build: status=".status.build"

Conversation

v1v commented Nov 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

Tests

Uh oh!

elasticmachine commented Nov 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Uh oh!

v1v commented Nov 10, 2020

Uh oh!

kuisathaverat Nov 10, 2020

Choose a reason for hiding this comment

Uh oh!

kuisathaverat Nov 10, 2020

Choose a reason for hiding this comment

Uh oh!

v1v Nov 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kuisathaverat Nov 10, 2020

Choose a reason for hiding this comment

Uh oh!

v1v Nov 10, 2020

Choose a reason for hiding this comment

Uh oh!

v1v commented Nov 11, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

v1v commented Nov 10, 2020 •

edited

Loading

elasticmachine commented Nov 10, 2020 •

edited

Loading

v1v Nov 10, 2020 •

edited

Loading