Skip to content

Created test_release.sh to test centos|fedora|ubuntu images #613

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tsachiherman merged 12 commits into from
Dec 12, 2019
Merged

Created test_release.sh to test centos|fedora|ubuntu images #613

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
e4d5bd3
Created `test_release.sh` to test centos|fedora|ubuntu images
Dec 5, 2019
ab43bad
Incorporate some review suggestions (more to come):
Dec 6, 2019
d8f15e2
Add ability to pass bucket, channel and aws creds
Dec 7, 2019
fc0012e
Ensure aws creds are in env before starting
Dec 9, 2019
ca01487
Make colorized text more readable
Dec 9, 2019
9fe7083
Break script into `build` and `run` operations
Dec 9, 2019
d51a038
Run `update.sh` at RUN time
Dec 9, 2019
c6bf2b1
Merge branch 'master' into test_releases
Dec 10, 2019
9ace86f
We're not writing the Dockerfile to disk before running it.
Dec 10, 2019
65b8383
Added new `post_deploy` stage and our script
Dec 10, 2019
0d52edb
Adding new `scripts/travis/test_release.sh` script
Dec 11, 2019
878e419
Add filtering for new `post_deploy` stage
Dec 11, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions scripts/test_release.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
#!/usr/bin/env bash

OS_LIST=(
centos:7
centos:8
fedora:28
ubuntu:16.04
ubuntu:18.04
)

FAILED=()
RET_VALUE=0

# We'll use this simple tokenized Dockerfile.
# https://serverfault.com/a/72511
IFS='' read -r -d '' TOKENIZED <<"EOF"
FROM {{OS}}

@winder winder Dec 6, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you included an ARG channel option you could specify different channels to this script. In update.sh you would use $channel This would let us run this test with stable/nightly/beta release channels.

@btoll btoll Dec 7, 2019

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

ENV DEBIAN_FRONTEND noninteractive
{{PACMAN}}
ADD https://algorand-releases.s3.us-east-1.amazonaws.com/channel/stable/install_stable_linux-amd64_2.0.1.tar.gz /tmp

RUN \
set -eux; \
mkdir /opt/installer ; \
cd /opt/installer ; \
tar xvf /tmp/install*tar.gz ; \
./update.sh -i -c stable -p /opt/algorand/node -d /opt/algorand/node/data -n ;

WORKDIR /opt/algorand/node
RUN ["./goal", "node", "start", "-d", "data"]

@winder winder Dec 6, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if you replaced this RUN with CMD ["/opt/algorand/node/algod", "-v"] you wouldn't need to specify the bash command when starting the node. Also you shouldn't need to start the node for this test.

@btoll btoll Dec 6, 2019

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to know, will change.

EOF

for item in ${OS_LIST[*]}
do
# Install root certs.
# We use pattern substitution here (like sed).
# ${parameter/pattern/substitution}
if [[ $item =~ ubuntu ]]
then
WITH_PACMAN=$(echo -e "${TOKENIZED//\{\{PACMAN\}\}/RUN apt update && apt install -y ca-certificates}")

@tsachiherman tsachiherman Dec 6, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use apt-get instead of apt

@btoll btoll Dec 6, 2019

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed.

@winder winder Dec 6, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can include the debian var as a one liner, I've seen it like this before:
RUN DEBIAN_FRONTEND=noninteractive apt update && apt install -y ca-certificates

@btoll btoll Dec 6, 2019

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, changed.

else
# CentOS/Fedora must have the updated root certs already installed.
WITH_PACMAN=$(echo -e "${TOKENIZED//\{\{PACMAN\}\}/}")
fi

# Finally, designate the OS and send the fully-formed Dockerfile to Docker.
echo -e "${WITH_PACMAN/\{\{OS\}\}/$item}" | docker build -t "$item" -

@winder winder Dec 6, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this cache the image if you run it multiple times?

@btoll btoll Dec 9, 2019

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it appears to be getting the image from the cache:

[./test_release.sh] Testing ubuntu:18.04...
Sending build context to Docker daemon   2.56kB
Step 1/8 : FROM ubuntu:18.04
 ---> 775349758637
Step 2/8 : WORKDIR /root/install
 ---> Using cache
 ---> 92c6605eb1f7
Step 3/8 : RUN DEBIAN_FRONTEND=noninteractive apt-get update && apt-get install -y curl ca-certificates --no-install-recommends
 ---> Using cache
 ---> e087dcbfd3fc
Step 4/8 : ENV AWS_ACCESS_KEY_ID=AKIAYH3ME2LO3I3T2KJ4
 ---> Using cache
 ---> c64da447d5f0
Step 5/8 : ENV AWS_SECRET_ACCESS_KEY=49gn/2FbwwjdzeazIBBbF7Xu1Otd7aRJAeCE3Um+
 ---> Using cache
 ---> ac6191ef03f1
Step 6/8 : RUN curl --silent -L https://github.com/algorand/go-algorand-doc/blob/master/downloads/installers/linux_amd64/install_master_linux-amd64.tar.gz?raw=true | tar xzf - &&         ./update.sh -b algorand-builds -c stable -n -p ~/node -d ~/node/data -i &&         cd .. &&         rm -rf install /var/lib/apt/lists/*
 ---> Using cache
 ---> d342ad4d99c6
Step 7/8 : WORKDIR /root/node
 ---> Using cache
 ---> 9b1e8ff9a556
Step 8/8 : CMD ["/bin/bash"]
 ---> Using cache
 ---> bfc71bf3a94b
Successfully built bfc71bf3a94b
Successfully tagged ubuntu:18.04-test

@winder winder Dec 9, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need to make sure that running the command multiple times can test different versions. Moving the update.sh call into the CMD might be one way to do that, assuming the CMD is executed every time you start the container.

I'm also wondering about the goal of test in general. @tsachiherman is the updater functionality something you're interested as part of this test, or just that the software can run? Perhaps we skip update.sh and just copy algod to the container and run algod -v?


if ! docker run -it "$item" /bin/bash -c "/opt/algorand/node/algod -v"
then
RET_VALUE=1
FAILED+=("$item")
fi
done

if [ "${#FAILED[@]}" -gt 0 ]
then
echo -e "\n$(tput setaf 1)[$0]$(tput sgr0) The following images have problems:"
for failed in ${FAILED[*]}
do
echo " - $failed"
done
echo
fi

exit $RET_VALUE