Skip to content

[ZEPPELIN-1386] Docker images for running Apache Zeppelin releases #1538

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mfelgamal wants to merge 12 commits into from
Closed

[ZEPPELIN-1386] Docker images for running Apache Zeppelin releases #1538

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
48d0a01
Merge pull request #1 from apache/master
mfelgamal Sep 7, 2016
ac06f3a
Make docker image for zeppelin release
mfelgamal Sep 24, 2016
231a414
Add zeppelin-base image
mfelgamal Oct 18, 2016
e642309
Add documentation and some modifications
mfelgamal Oct 21, 2016
e731cb4
Add java-cacerts to zeppelin-base
mfelgamal Oct 23, 2016
e1d4b77
add R and python to zeppelin-base
mfelgamal Nov 3, 2016
fd23970
remove bash erorr message.
mfelgamal Nov 5, 2016
d2c744e
add scala to zeppelin-base
mfelgamal Nov 12, 2016
1f093d4
Add script start-zeppelin to zeppelin-base
mfelgamal Nov 18, 2016
b64c680
Remove gcc and g++ for decreasing the size
mfelgamal Dec 1, 2016
d48ecef
fix: Remove start-zeppelin.sh
1ambda Dec 13, 2016
cc8493f
Merge pull request #3 from 1ambda/fix/remove-startzeppelinsh
mfelgamal Dec 13, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 24 additions & 4 deletions dev/create_release.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,14 +33,28 @@ if [[ $# -ne 2 ]]; then
usage
fi

if [[ -z "${GPG_PASSPHRASE}" ]]; then
echo "You need GPG_PASSPHRASE variable set"
exit 1
fi
for var in GPG_PASSPHRASE DOCKER_USERNAME; do
if [[ -z "${!var}" ]]; then
echo "You need ${var} variable set"
exit 1
fi
done

RELEASE_VERSION="$1"
GIT_TAG="$2"

function build_docker_base() {
# build base image
docker build -t ${DOCKER_USERNAME}/zeppelin-base:latest "${WORKING_DIR}/zeppelin/scripts/docker/zeppelin-base"
}
function build_docker_image() {
# build release image
echo "FROM ${DOCKER_USERNAME}/zeppelin-base:latest
RUN mkdir /usr/local/zeppelin/
ADD zeppelin-${RELEASE_VERSION}-bin-${BIN_RELEASE_NAME} /usr/local/zeppelin/" > "Dockerfile"
docker build -t ${DOCKER_USERNAME}/zeppelin-release:"${RELEASE_VERSION}" .
}

function make_source_package() {
# create source package
cd ${WORKING_DIR}
Expand Down Expand Up @@ -97,12 +111,18 @@ function make_binary_release() {
mv "zeppelin-${RELEASE_VERSION}-bin-${BIN_RELEASE_NAME}.tgz.md5" "${WORKING_DIR}/"
mv "zeppelin-${RELEASE_VERSION}-bin-${BIN_RELEASE_NAME}.tgz.sha512" "${WORKING_DIR}/"

# build docker image if binary_release_name 'all'
if [[ $1 = "all" ]]; then
build_docker_image
fi

# clean up build dir
rm -rf "${WORKING_DIR}/zeppelin-${RELEASE_VERSION}-bin-${BIN_RELEASE_NAME}"
}

git_clone
make_source_package
build_docker_base
make_binary_release all "-Pspark-2.0 -Phadoop-2.4 -Pyarn -Ppyspark -Psparkr -Pr -Pscala-2.11"
make_binary_release netinst "-Pspark-2.0 -Phadoop-2.4 -Pyarn -Ppyspark -Psparkr -Pr -Pscala-2.11 -pl !alluxio,!angular,!cassandra,!elasticsearch,!file,!flink,!hbase,!ignite,!jdbc,!kylin,!lens,!livy,!markdown,!postgresql,!python,!shell,!bigquery"

Expand Down
11 changes: 10 additions & 1 deletion dev/publish_release.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ if [[ $# -ne 2 ]]; then
usage
fi

for var in GPG_PASSPHRASE ASF_USERID ASF_PASSWORD; do
for var in GPG_PASSPHRASE ASF_USERID ASF_PASSWORD DOCKER_USERNAME DOCKER_PASSWORD DOCKER_EMAIL; do
if [[ -z "${!var}" ]]; then
echo "You need ${var} variable set"
exit 1
Expand Down Expand Up @@ -67,6 +67,14 @@ function curl_error() {
fi
}

function publish_to_dockerhub() {
# publish images
docker login --username="${DOCKER_USERNAME}" --password="${DOCKER_PASSWORD}" --email="${DOCKER_EMAIL}"
docker push ${DOCKER_USERNAME}/zeppelin-base:latest
docker push ${DOCKER_USERNAME}/zeppelin-release:"${RELEASE_VERSION}"

}

function publish_to_maven() {
cd "${WORKING_DIR}/zeppelin"

Expand Down Expand Up @@ -153,5 +161,6 @@ function publish_to_maven() {
}

git_clone
publish_to_dockerhub
publish_to_maven
cleanup
16 changes: 16 additions & 0 deletions scripts/docker/zeppelin-base/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
FROM alpine:3.3
MAINTAINER Mahmoud Elgamal <mahmoudf.elgamal@gmail.com>

@bzz bzz Oct 19, 2016 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the top of my head:

  • there must be ASF Apache 2.0 license header at the beginning of every file
  • ASF projects usually discourage author annotations, could you please replace this with something like Apache Zeppelin authors <dev@zeppelin.apache.org>?

Attributions are kept though JIRA\Git commit logs.

@felixcheung felixcheung Oct 21, 2016

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

usually people put Apache Software Foundation <dev@zeppelin.apache.org>

@mfelgamal mfelgamal Oct 21, 2016

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


RUN apk add --update bash curl openjdk7-jre wget ca-certificates openssl && rm -rf /var/cache/apk/*

RUN wget -O /usr/local/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v1.1.3/dumb-init_1.1.3_amd64
RUN chmod +x /usr/local/bin/dumb-init

# set jave environment variable
ENV JAVA_HOME /usr/lib/jvm/java-1.7-openjdk
ENV PATH $PATH:$JAVA_HOME/bin

# ports for zeppelin
EXPOSE 8080 8081

@anthonycorbacho anthonycorbacho Oct 22, 2016

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in Zeppelin we dont need to open the port 8081 (used for old implementation of websocket), only 8080 is required now

@mfelgamal mfelgamal Oct 23, 2016

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@anthonycorbacho Ok, so if you want to build a docker image to the old version of zeppelin, you need port 8081, what do you think about?

@anthonycorbacho anthonycorbacho Oct 27, 2016

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mfelgamal you are right, but the usage of port 8081 have been removed long time ago, do you think it is worth to keep it? I am note sure if ppl will sill want to use an relic version of zeppelin?
Maybe by nostalgia ?

Let me know if it make sense :)

@mfelgamal mfelgamal Nov 3, 2016

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@anthonycorbacho you are right, I removed 8081 port :)


ENTRYPOINT ["/usr/local/bin/dumb-init"]

@1ambda 1ambda Nov 30, 2016 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be ENTRYPOINT ["/usr/local/bin/dumb-init", "--"] according to https://github.com/Yelp/dumb-init#usage. It allows extened images run their executables without specifying -- if fixed

@mfelgamal mfelgamal Dec 1, 2016

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@1ambda done.