Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Heroku docker deploy #1301

Merged
merged 47 commits into from
Apr 19, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
7afede6
attempting to deploy static assets with heroku
metasoarous Sep 24, 2021
e07ecba
switch to pre-built babashka images
metasoarous Sep 24, 2021
7ab883e
fix script call
metasoarous Sep 25, 2021
29c2ae5
update script deps
metasoarous Sep 27, 2021
9ed49fe
add docker and jvm to build dockerfile
metasoarous Sep 27, 2021
d899ee2
update static asset deploy script to use current aws env variables
metasoarous Sep 29, 2021
b5b0295
remove amazonica in favor of cognitect/aws-cli lib
metasoarous Oct 14, 2021
516bedc
add POLLER_ZID_BLOCKLIST config setting
metasoarous Oct 14, 2021
18ed2a8
temporarily turn off release phase in heroku.yml
metasoarous Oct 14, 2021
59517b9
update postgresql pod in purge-pii script
metasoarous Oct 14, 2021
6f4600a
some docker-dev.env documentation
metasoarous Oct 14, 2021
16073ec
Modify main Dockerfile to try and copy built assets from other contai…
metasoarous Oct 14, 2021
cf6c58e
Partially fix build for lima and podman
willcohen Oct 15, 2021
e1b71b5
modify heroku.yml to build client containers
metasoarous Oct 18, 2021
dd3b4e8
try renaming client images to match FROM clauses in deploy deockerfile
metasoarous Oct 18, 2021
e9b8c92
fix typo in image names in heroku.yml
metasoarous Oct 18, 2021
ba4b289
Merge pull request #1135 from willcohen/podman
metasoarous Nov 4, 2021
c8aaf7b
Merge dockerfiles
metasoarous Nov 5, 2021
d2be1fb
update heroku.yml to use file-server/Dockerfile
metasoarous Nov 5, 2021
01af2d4
move work from file-server/Dockerfile to project root for heroku
metasoarous Nov 5, 2021
6e5fd79
update deploy-static-assets path
metasoarous Nov 5, 2021
7865119
update tos to point to compdemocracy.org addresses
metasoarous Nov 5, 2021
a616641
Reset repo
metasoarous Nov 5, 2021
d89ac48
adding babashka to release container
metasoarous Nov 5, 2021
1a15740
fixed asets paths in static asset build & deploy docker image
metasoarous Jan 31, 2022
c4a779b
simplify --bucket config option to static asset deploy script
metasoarous Jan 31, 2022
b7e8b09
remove explicit command call from heroku.yml
metasoarous Jan 31, 2022
3e3417d
update tos & privacy doc emails and org name
metasoarous Jan 31, 2022
47382a4
cut out unecessary heroku.yml sections
metasoarous Jan 31, 2022
fd8d80a
re-add explicit release section to heroku.yml
metasoarous Jan 31, 2022
1a20e0f
re-add explicit release command to heroku.yml
metasoarous Jan 31, 2022
2cfdaa4
switch from entrypoint to cmd in dockerfile
metasoarous Jan 31, 2022
72a2207
fix deploy command invocation path in heroku.yml
metasoarous Jan 31, 2022
204f0c7
whitespace cleanup (to trigger redeploy)
metasoarous Feb 1, 2022
7b5a4b7
update pviracy and tos dates to reflect changes to company name
metasoarous Mar 8, 2022
d9818db
resolve merge conflicts with dev
metasoarous Mar 8, 2022
6ab90cd
remove build:watch command from Dockerfile to docker-compose.dev.yml …
metasoarous Apr 12, 2022
fc95762
remove port exposure from server/Docker and into docker-compose.dev.yml
metasoarous Apr 12, 2022
61f9c46
Merge remote-tracking branch 'origin/dev' into heroku-deploy-static-a…
metasoarous Apr 12, 2022
060a78d
separate build and run phases in server/Dockerfile
metasoarous Apr 13, 2022
3ef7b94
fix pca result caching in server; fixes #1380
metasoarous Apr 16, 2022
0143e22
revert to older node for admin-client in file-server/Dockerfile
metasoarous Apr 16, 2022
81cb6a0
fix paths in file-server
metasoarous Apr 19, 2022
7d6db8e
comment out task-system in full-system for now; fixes #1279
metasoarous Apr 19, 2022
a319f58
Add package-lock.json files
metasoarous Apr 19, 2022
e32375c
add one more package-lock.json file
metasoarous Apr 19, 2022
c927b5f
Cleanup up file-server/Dockerfile and fix npm ci issue
metasoarous Apr 19, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 86 additions & 10 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,14 +1,90 @@
FROM alpine:latest
ARG TAG=dev

ENV BUILD_DEPS="bash build-base libpng-dev zlib-dev autoconf automake libtool nasm curl" \
RUN_DEPS="openssh-client openjdk8-jre"
# polis-client-admin
# Gulp v3 stops us from upgrading beyond Node v11
FROM docker.io/node:11.15.0-alpine

WORKDIR /build
COPY . .
WORKDIR /client-admin/app

# for build scripting
RUN \
bash < <(curl -s https://raw.githubusercontent.com/babashka/babashka/master/install)
RUN apk add git --no-cache

COPY client-admin/package*.json ./
RUN npm install

COPY client-admin/polis.config.template.js polis.config.js
# If polis.config.js exists on host, will override template here.
COPY client-admin/. .

ARG GIT_HASH
RUN npm run deploy:prod


# polis-client-participation
# # Gulp v3 stops us from upgrading beyond Node v11
FROM docker.io/node:11.15.0-alpine

WORKDIR /client-participation/app

RUN apk add --no-cache --virtual .build \
g++ git make python

# Allow global npm installs in Docker
RUN npm config set unsafe-perm true

# Upgrade npm v6.7.0 -> v6.9.2 to alias multiple pkg versions.
# See: https://stackoverflow.com/a/56134858/504018
RUN npm install -g [email protected]

COPY client-participation/package*.json ./

RUN npm ci

RUN apk del .build

COPY client-participation/polis.config.template.js polis.config.js
# If polis.config.js exists on host, will override template here.
COPY client-participation/. .

ARG GIT_HASH
ARG BABEL_ENV=production

RUN npm run deploy:prod


# polis-client-report
# Gulp v3 stops us from upgrading beyond Node v11
FROM docker.io/node:11.15.0-alpine

WORKDIR /client-report/app

RUN apk add git --no-cache

COPY client-report/package*.json ./
RUN npm ci

COPY client-report/polis.config.template.js polis.config.js
# If polis.config.js exists on host, will override template here.
COPY client-report/. .

ARG GIT_HASH
RUN npm run deploy:prod


#FROM docker.io/node:16.9.0-alpine
FROM docker.io/babashka/babashka

RUN apt-get update && apt-get -y install openjdk-16-jre


RUN mkdir /app/

WORKDIR /app/

COPY ./bin/ ./bin/

COPY --from=0 /client-admin/app/dist/ /app/client-admin/dist
COPY --from=1 /client-participation/app/build/ /app/client-participation/dist
COPY --from=2 /client-report/app/build/ /app/client-report/dist

CMD ./bin/deploy-static-assets.clj --bucket $STATIC_ASSET_DEPLOY_BUCKET

#apk del $BUILD_DEPS && \
#rm -rf /tmp/* /var/cache/*
52 changes: 32 additions & 20 deletions bin/deploy-static-assets.clj
Original file line number Diff line number Diff line change
Expand Up @@ -9,22 +9,28 @@
'[clojure.java.io :as io]
'[clojure.string :as string])

(pods/load-pod 'org.babashka/postgresql "0.0.1")
(pods/load-pod 'org.babashka/aws "0.0.5")
(pods/load-pod 'org.babashka/postgresql "0.0.7")
(pods/load-pod 'org.babashka/aws "0.0.6")
(deps/add-deps '{:deps {honeysql/honeysql {:mvn/version "1.0.444"}}})

(require '[pod.babashka.postgresql :as pg]
'[honeysql.core :as hsql]
'[honeysql.helpers :as hsqlh]
'[pod.babashka.aws :as aws])
'[pod.babashka.aws :as aws]
'[pod.babashka.aws.credentials :as aws-creds])

;; Should move this to arg parsing if and when available
(def region (or (System/getenv "AWS_REGION")
"us-east-1"))

(def creds-provider
(aws-creds/basic-credentials-provider
{:access-key-id (System/getenv "AWS_ACCESS_KEY")
:secret-access-key (System/getenv "AWS_SECRET_KEY")}))

(def s3-client
"The s3 client for this process"
(aws/client {:api :s3 :region region}))
(aws/client {:api :s3 :region region :credentials-provider creds-provider}))

;; list available s3 actions
;(map first (aws/ops s3-client))
Expand Down Expand Up @@ -96,7 +102,7 @@

(defn latest-version [cache-dir]
(->> (.listFiles (io/file cache-dir))
(sort-by #(mapv try-int (string/split % #"_")))
(sort-by #(mapv try-int (string/split (str %) #"_")))
(last)))

;(latest-version "participation-client/dist/cached")
Expand Down Expand Up @@ -205,15 +211,14 @@

(def concurrent-requests 12)

(println "arg" (pr-str (first *command-line-args*)))
(def bucket
(case (first *command-line-args*)
(defn get-bucket [bucket-arg]
(case bucket-arg
"PRODUCTION" "pol.is"
;; else
"preprod.pol.is"))

("preprod" "edge") "preprod.pol.is"
bucket-arg))

(def responses
(defn responses [bucket]
(let [requests (mapcat (partial spec-requests bucket)
deploy-specs)
output-chan (async/chan concurrent-requests)]
Expand All @@ -223,18 +228,25 @@
(async/to-chan requests))
(async/<!! (async/into [] output-chan))))

(def errors
(defn errors [responses]
(remove (comp :ETag second)
responses))

(if (not-empty errors)
(do
(println "Problem processing" (count errors) "requests")
(doseq [[request response] errors]
(println "Problem processing request:" request)
(pp/pprint response)))
(println "Deploy completed without error."))

(defn -main [& {:as opts-map :strs [--bucket]}]
(let [bucket (get-bucket --bucket)
_ (println "Deploying static assets to bucket:" bucket)
responses (responses bucket)
errors (errors responses)]
(if (not-empty errors)
(do
(println "Problem processing" (count errors) "requests")
(doseq [[request response] errors]
(println "Problem processing request:" request)
(pp/pprint response)))
(println "Deploy completed without error."))))


(apply -main *command-line-args*)

;; QED

2 changes: 1 addition & 1 deletion client-admin/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Gulp v3 stops us from upgrading beyond Node v11
FROM node:11.15.0-alpine
FROM docker.io/node:11.15.0-alpine

WORKDIR /app

Expand Down
Loading