Apache Storm 1.1.1 #3306

31z4 · 2017-08-10T05:22:00Z

For some reason I cannot build localy with the following error
gpg: keyserver receive failed: Address not available
I've tried to replace ha.pool.sks-keyservers.net with ipv4.pool.sks-keyservers.net and it worked. Is this the right thing to do?

tianon · 2017-08-10T17:39:31Z

So, this discussion has a bit of history. When we first started doing --recv-keys, we hard-coded the keyserver to pgp.mit.edu. This was great when the server was up, but it's a single server, so any time it had a hiccup, our builds would be cascading fails.

To combat that in the simplest way possible, we switched to ha.pool.sks-keyservers.net. The biggest benefit is that it represents a pool of many servers, so when one fails, it usually just requires a build restart and things generally work. Sometimes (and I think this is probably what you ran into, because I ran into similar issues yesterday) we actually see architectural issues in SKS, although they are more rare (bad rotations occasionally, but usually just DNS wiggles like I saw yesterday, where ha.pool.sks-keyservers.net became empty for a short while).

If you'd like to embed a somewhat more-permanent solution in your Dockerfile, you probably want to take a look at docker-solr/docker-solr#113 and/or docker-solr/docker-solr#114, nodejs/docker-node#375. The general idea is to try several different server names in sequence until one works, but the trick is to make sure that the build still fails if all the servers do. 👍

tianon · 2017-08-10T17:39:41Z

Diff:

diff --git a/_bashbrew-list b/_bashbrew-list
index f10bc22..8d19dbe 100644
--- a/_bashbrew-list
+++ b/_bashbrew-list
@@ -5,5 +5,5 @@ storm:0.10.2
 storm:1.0
 storm:1.0.3
 storm:1.1
-storm:1.1.0
+storm:1.1.1
 storm:latest
diff --git a/storm_0.10/Dockerfile b/storm_0.10/Dockerfile
index c851d57..642b3c7 100644
--- a/storm_0.10/Dockerfile
+++ b/storm_0.10/Dockerfile
@@ -1,5 +1,4 @@
 FROM openjdk:8-jre-alpine
-MAINTAINER Elisey Zanko <[email protected]>
 
 # Install required packages
 RUN apk add --no-cache \
@@ -7,10 +6,10 @@ RUN apk add --no-cache \
     python \
     su-exec
 
-ENV STORM_USER=storm
-ENV STORM_CONF_DIR=/conf
-ENV STORM_DATA_DIR=/data
-ENV STORM_LOG_DIR=/logs
+ENV STORM_USER=storm \
+    STORM_CONF_DIR=/conf \
+    STORM_DATA_DIR=/data \
+    STORM_LOG_DIR=/logs
 
 # Add a user and make dirs
 RUN set -x \
@@ -32,7 +31,7 @@ RUN set -x \
     && gpg --batch --verify "$DISTRO_NAME.tar.gz.asc" "$DISTRO_NAME.tar.gz" \
     && tar -xzf "$DISTRO_NAME.tar.gz" \
     && chown -R "$STORM_USER:$STORM_USER" "$DISTRO_NAME" \
-    && rm -r "$GNUPGHOME" "$DISTRO_NAME.tar.gz" "$DISTRO_NAME.tar.gz.asc" \
+    && rm -rf "$GNUPGHOME" "$DISTRO_NAME.tar.gz" "$DISTRO_NAME.tar.gz.asc" \
     && apk del .build-deps
 
 WORKDIR $DISTRO_NAME
diff --git a/storm_0.9/Dockerfile b/storm_0.9/Dockerfile
index 5830aee..37d0e8f 100644
--- a/storm_0.9/Dockerfile
+++ b/storm_0.9/Dockerfile
@@ -1,5 +1,4 @@
 FROM openjdk:8-jre-alpine
-MAINTAINER Elisey Zanko <[email protected]>
 
 # Install required packages
 RUN apk add --no-cache \
@@ -7,10 +6,10 @@ RUN apk add --no-cache \
     python \
     su-exec
 
-ENV STORM_USER=storm
-ENV STORM_CONF_DIR=/conf
-ENV STORM_DATA_DIR=/data
-ENV STORM_LOG_DIR=/logs
+ENV STORM_USER=storm \
+    STORM_CONF_DIR=/conf \
+    STORM_DATA_DIR=/data \
+    STORM_LOG_DIR=/logs
 
 # Add a user and make dirs
 RUN set -x \
@@ -32,7 +31,7 @@ RUN set -x \
     && gpg --batch --verify "$DISTRO_NAME.tar.gz.asc" "$DISTRO_NAME.tar.gz" \
     && tar -xzf "$DISTRO_NAME.tar.gz" \
     && chown -R "$STORM_USER:$STORM_USER" "$DISTRO_NAME" \
-    && rm -r "$GNUPGHOME" "$DISTRO_NAME.tar.gz" "$DISTRO_NAME.tar.gz.asc" \
+    && rm -rf "$GNUPGHOME" "$DISTRO_NAME.tar.gz" "$DISTRO_NAME.tar.gz.asc" \
     && apk del .build-deps
 
 WORKDIR $DISTRO_NAME
diff --git a/storm_1.0/Dockerfile b/storm_1.0/Dockerfile
index 339c6bf..086ca61 100644
--- a/storm_1.0/Dockerfile
+++ b/storm_1.0/Dockerfile
@@ -31,7 +31,7 @@ RUN set -x \
     && gpg --batch --verify "$DISTRO_NAME.tar.gz.asc" "$DISTRO_NAME.tar.gz" \
     && tar -xzf "$DISTRO_NAME.tar.gz" \
     && chown -R "$STORM_USER:$STORM_USER" "$DISTRO_NAME" \
-    && rm -r "$GNUPGHOME" "$DISTRO_NAME.tar.gz" "$DISTRO_NAME.tar.gz.asc" \
+    && rm -rf "$GNUPGHOME" "$DISTRO_NAME.tar.gz" "$DISTRO_NAME.tar.gz.asc" \
     && apk del .build-deps
 
 WORKDIR $DISTRO_NAME
diff --git a/storm_latest/Dockerfile b/storm_latest/Dockerfile
index c7b3aa6..2fff65d 100644
--- a/storm_latest/Dockerfile
+++ b/storm_latest/Dockerfile
@@ -18,7 +18,7 @@ RUN set -x \
     && chown -R "$STORM_USER:$STORM_USER" "$STORM_CONF_DIR" "$STORM_DATA_DIR" "$STORM_LOG_DIR"
 
 ARG GPG_KEY=ACEFE18DD2322E1E84587A148DE03962E80B8FFD
-ARG DISTRO_NAME=apache-storm-1.1.0
+ARG DISTRO_NAME=apache-storm-1.1.1
 
 # Download Apache Storm, verify its PGP signature, untar and clean up
 RUN set -x \
@@ -31,7 +31,7 @@ RUN set -x \
     && gpg --batch --verify "$DISTRO_NAME.tar.gz.asc" "$DISTRO_NAME.tar.gz" \
     && tar -xzf "$DISTRO_NAME.tar.gz" \
     && chown -R "$STORM_USER:$STORM_USER" "$DISTRO_NAME" \
-    && rm -r "$GNUPGHOME" "$DISTRO_NAME.tar.gz" "$DISTRO_NAME.tar.gz.asc" \
+    && rm -rf "$GNUPGHOME" "$DISTRO_NAME.tar.gz" "$DISTRO_NAME.tar.gz.asc" \
     && apk del .build-deps
 
 WORKDIR $DISTRO_NAME

tianon · 2017-08-10T17:44:05Z

LGTM

Build test of #3306; cd3e64a (storm):

$ bashbrew build storm:0.9.7
Building bashbrew/cache:8bfd33321df436f02ed9de297be103e34731a6d12fb32c93c7df6ecd6f3557fd (storm:0.9.7)
Tagging storm:0.9.7
Tagging storm:0.9

$ test/run.sh storm:0.9.7
testing storm:0.9.7
	'utc' [1/4]...passed
	'cve-2014--shellshock' [2/4]...passed
	'no-hard-coded-passwords' [3/4]...passed
	'override-cmd' [4/4]...passed


$ bashbrew build storm:0.10.2
Building bashbrew/cache:5bcf98170db33d2562525125847d29c51842cdac3608c780c151db7a79b071e6 (storm:0.10.2)
Tagging storm:0.10.2
Tagging storm:0.10

$ test/run.sh storm:0.10.2
testing storm:0.10.2
	'utc' [1/4]...passed
	'cve-2014--shellshock' [2/4]...passed
	'no-hard-coded-passwords' [3/4]...passed
	'override-cmd' [4/4]...passed


$ bashbrew build storm:1.0.3
Building bashbrew/cache:ceb90ab8a57460567bda4429ccca8bbc0dd54f8f7a152a4dc7d18ee1056ea8a7 (storm:1.0.3)
Tagging storm:1.0.3
Tagging storm:1.0

$ test/run.sh storm:1.0.3
testing storm:1.0.3
	'utc' [1/4]...passed
	'cve-2014--shellshock' [2/4]...passed
	'no-hard-coded-passwords' [3/4]...passed
	'override-cmd' [4/4]...passed


$ bashbrew build storm:1.1.1
Building bashbrew/cache:a46c5d8bd4032d5e05f804d3a47bf038fe9659eed2d51270f3e97421fffc0d41 (storm:1.1.1)
Tagging storm:1.1.1
Tagging storm:1.1
Tagging storm:latest

$ test/run.sh storm:1.1.1
testing storm:1.1.1
	'utc' [1/4]...passed
	'cve-2014--shellshock' [2/4]...passed
	'no-hard-coded-passwords' [3/4]...passed
	'override-cmd' [4/4]...passed

31z4 · 2017-08-11T03:54:43Z

Thanks for the explanation @tianon! ❤️

Apache Storm 1.1.1

cd3e64a

docker-library-bot added the library/storm label Aug 10, 2017

tianon merged commit 8a4216d into docker-library:master Aug 10, 2017

yosifkit mentioned this pull request Aug 10, 2017

Timeouts with Keyserver | Debating on providing key in build context #3312

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Apache Storm 1.1.1 #3306

Apache Storm 1.1.1 #3306

31z4 commented Aug 10, 2017

tianon commented Aug 10, 2017

tianon commented Aug 10, 2017

tianon commented Aug 10, 2017

31z4 commented Aug 11, 2017

Apache Storm 1.1.1 #3306

Apache Storm 1.1.1 #3306

Conversation

31z4 commented Aug 10, 2017

tianon commented Aug 10, 2017

tianon commented Aug 10, 2017

tianon commented Aug 10, 2017

31z4 commented Aug 11, 2017