Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/on-prerelease.yml
Original file line number Diff line number Diff line change
Expand Up @@ -245,7 +245,7 @@ jobs:
$arguments.PackageFile = $_
./build.ps1 Push @arguments
}

docker-publish:
name: Publish to Docker Hub
runs-on: ubuntu-latest
Expand All @@ -259,10 +259,10 @@ jobs:
- name: Prepare Tags
id: prepare-tags
run: |
REF="${{ env.REF }}"
REF="${{ env.REF }}"
if [[ $REF =~ "Pre-Release" ]]
then
# Remove Pre-Release prefix
# Remove Pre-Release prefix
PREFIX="Pre-Release-"
PACKAGEVERSION=${REF#"$PREFIX"}
else
Expand All @@ -286,7 +286,7 @@ jobs:
SEMVERSION=${PACKAGEVERSION:1} # strip off the leading 'v'
echo "APITAGS=$APITAGS" >> $GITHUB_OUTPUT
echo "DBTAGS=$DBTAGS" >> $GITHUB_OUTPUT
echo "VERSION=$SEMVERSION" >> $GITHUB_OUTPUT
echo "VERSION=$SEMVERSION" >> $GITHUB_OUTPUT

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@4b4e9c3e2d4531116a6f8ba8e71fc6e2cb6e6c8c # v2.5.0
Expand All @@ -310,7 +310,7 @@ jobs:
cache-from: type=registry,ref=${{ env.IMAGE_NAME }}:pre-1.3
cache-to: type=inline
build-args: VERSION=${{ steps.prepare-tags.outputs.VERSION }}
file: pgsql.Dockerfile
file: Dockerfile
tags: ${{ steps.prepare-tags.outputs.APITAGS }}
labels: ${{ steps.metaapi.outputs.labels }}
push: true
Expand Down
86 changes: 74 additions & 12 deletions .github/workflows/on-pullrequest-dockerfile.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,6 @@ on:
push:
branches:
- main
paths:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This work is being triggered only when the PR includes a change into the Dockerfiles or the action, the issue with this is that it can cause the analysis to go without running for a while, therefore, if there are new vulnerabilities it can go unnoticed.

I'm adding this run after merging the PRs to avoid adding noise to the PR itself since current behavior only shows the status, it does not forces or fails the build when new vulnerabilities appear.

- ".github/workflows/on-pullrequest-dockerfile.yml"
- "Docker/*"
pull_request:
branches:
- main
Expand All @@ -20,25 +17,90 @@ on:
- "Docker/*"
workflow_dispatch:

env:
DOCKER_USERNAME: ${{ vars.DOCKER_USERNAME }}
DOCKER_HUB_TOKEN: ${{ secrets.DOCKER_HUB_TOKEN }}

jobs:
docker-testing:
docker-analysis:
runs-on: ubuntu-latest
permissions:
security-events: write
pull-requests: write
strategy:
fail-fast: false
matrix:
dockerfile: [
{ name: "ApiDatabase", path: "Docker/Settings/DB-Admin/pgsql/Dockerfile" },
{ name: "Postgres", path: "Docker/pgsql.Dockerfile" },
{ name: "Sql", path: "Docker/mssql.Dockerfile" },
{ name: "Database", path: "Docker/dbadmin.Dockerfile" },
{ name: "Development", path: "Docker/dev.Dockerfile" },
dockerfile:
[
{ name: "api-database", path: "Docker/Settings/DB-Admin/pgsql/Dockerfile", type: "published" },
{ name: "postgres", path: "Docker/Dockerfile", type: "published" },
{ name: "database", path: "Docker/dbadmin.Dockerfile", type: "local" },
{ name: "development", path: "Docker/dev.Dockerfile", type: "local" },
]
steps:
- name: Checkout code
uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # v4.1.0
with:
fetch-depth: 0w

- name: Copy application folder to docker context
if: ${{ matrix.dockerfile.type == 'local' }}
run: |
mkdir Docker/Application
cp -r ./Application/EdFi.Ods.AdminApi ./Docker/Application
cp ./Application/NuGet.Config ./Docker/Application

- name: Set Version Numbers
if: ${{ matrix.dockerfile.type == 'published' }}
id: versions
run: |
tag=$(git describe --tags $(git rev-list --tags --max-count=1))

if [[ $tag =~ "Pre-Release" ]]
then
# Remove Pre-Release prefix
prefix="Pre-Release-"
package=${tag#"$prefix"}
else
package=${tag}
fi

SEMVERSION=${package:1} # strip off the leading 'v'
echo "Current version: $SEMVERSION"
echo "VERSION=$SEMVERSION" >> $GITHUB_OUTPUT

- uses: hadolint/hadolint-action@54c9adbab1582c2ef04b2016b760714a4bfde3cf # v3.1.0
name: Run Linter on ${{ matrix.dockerfile.name }} Dockerfile
with:
dockerfile: ${{ matrix.dockerfile.path }}
failure-threshold: error
failure-threshold: error

- name: Log in to Docker Hub
uses: docker/login-action@343f7c4344506bcbf9b4de18042ae17996df046d # v3.0.0
with:
username: ${{ env.DOCKER_USERNAME }}
password: ${{ env.DOCKER_HUB_TOKEN }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docker Scout requires authorization? I suppose that means we get only a few (was it 3?) free repositories this way.

Are we able to just run docker scout cves --format sarif --output scout.json [local image name] at the command line without needing Docker Hub credentials?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tried it and did not work. Must have Docker Hub login. Please help me understand what we get for free and what we would need to pay for.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was confusing for me too, basically, to use Docker Scout we need to be logged in but does not count into the 3 allowed repositories. The limit is only for the website enabled ones (that provides some graphs and additional information).

This repo is running for 4 containers, and none of those containers are enabled in docker scout and two of those not even exist in docker hub.
https://github.com/Ed-Fi-Alliance-OSS/Ed-Fi-AdminAPI/actions/runs/7050537942


- name: Build
run: |
path=${{matrix.dockerfile.path}}
folder=${path%/*}
cd $folder
dockerfile=$(echo ${{matrix.dockerfile.path}} | awk -F"/" '{print $NF}')

docker build -f $dockerfile -t ${{ matrix.dockerfile.name }} --build-arg="VERSION=${{ steps.versions.outputs.VERSION }}" .

- name: Analyze
uses: docker/scout-action@7c61653c2736d21969dd4593fde76c670d4a86cb # v1.2.0
with:
command: cves
image: local://${{ matrix.dockerfile.name }}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adding local:// to avoid using published image in case local is not found

sarif-file: sarif-${{ matrix.dockerfile.name }}.output.json
summary: true

- name: Upload SARIF result
id: upload-sarif
if: ${{ github.event_name != 'pull_request_target' }}
uses: github/codeql-action/upload-sarif@df32e399139a3050671466d7d9b3cbacc1cfd034 #codeql-bundle-v2.15.2
with:
sarif_file: sarif-${{ matrix.dockerfile.name }}.output.json
2 changes: 1 addition & 1 deletion Docker/Compose/pgsql/compose-build-binaries.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ services:
adminapi:
build:
context: ../../
dockerfile: pgsql.Dockerfile
dockerfile: Dockerfile
environment:
ADMIN_POSTGRES_HOST: db-admin
POSTGRES_PORT: 5432
Expand Down
10 changes: 6 additions & 4 deletions Docker/mssql.Dockerfile → Docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,22 +3,24 @@
# The Ed-Fi Alliance licenses this file to you under the Apache License, Version 2.0.
# See the LICENSE and NOTICES files in the project root for more information.

#tag 6.0-alpine
#tag 6.0-alpine
FROM mcr.microsoft.com/dotnet/aspnet@sha256:201cedd60cb295b2ebea7184561a45c5c0ee337e37300ea0f25cff5a2c762538
LABEL maintainer="Ed-Fi Alliance, LLC and Contributors <techsupport@ed-fi.org>"
ARG VERSION=latest
ARG DB=pgsql
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default value for the DB is pgsql, can be changed to mssql


# Alpine image does not contain Globalization Cultures library so we need to install ICU library to get for LINQ expression to work
# Disable the globaliztion invariant mode (set in base image)
ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false

WORKDIR /app

COPY Settings/mssql/appsettings.template.json /app/appsettings.template.json
COPY Settings/mssql/run.sh /app/run.sh
COPY Settings/mssql/log4net.config /app/log4net.txt
COPY Settings/"${DB}"/appsettings.template.json /app/appsettings.template.json
COPY Settings/"${DB}"/run.sh /app/run.sh
COPY Settings/"${DB}"/log4net.config /app/log4net.txt

RUN apk --no-cache add curl=~8 unzip=~6 dos2unix=~7 bash=~5 gettext=~0 jq=~1 icu=~72 && \
if [ "$DB" = "pgsql" ]; then apk --no-cache add postgresql13-client=~13; fi && \
wget -nv -O /app/AdminApi.zip https://pkgs.dev.azure.com/ed-fi-alliance/Ed-Fi-Alliance-OSS/_apis/packaging/feeds/EdFi/nuget/packages/EdFi.Suite3.ODS.AdminApi/versions/${VERSION}/content && \
unzip /app/AdminApi.zip AdminApi/* -d /app/ && \
cp -r /app/AdminApi/. /app/ && \
Expand Down
2 changes: 1 addition & 1 deletion Docker/Settings/DB-Admin/pgsql/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ ARG VERSION=latest
COPY run-adminapi-migrations.sh /docker-entrypoint-initdb.d/3-run-adminapi-migrations.sh

RUN apk --no-cache add dos2unix=~7.4 unzip=~6.0 && \
wget -O /tmp/EdFi_AdminApi_Scripts.zip https://pkgs.dev.azure.com/ed-fi-alliance/Ed-Fi-Alliance-OSS/_apis/packaging/feeds/EdFi/nuget/packages/EdFi.Suite3.ODS.AdminApi/versions/${VERSION}/content && \
wget -nv -O /tmp/EdFi_AdminApi_Scripts.zip https://pkgs.dev.azure.com/ed-fi-alliance/Ed-Fi-Alliance-OSS/_apis/packaging/feeds/EdFi/nuget/packages/EdFi.Suite3.ODS.AdminApi/versions/${VERSION}/content && \
unzip /tmp/EdFi_AdminApi_Scripts.zip AdminApi/Artifacts/PgSql/Structure/Admin/* -d /tmp/AdminApiScripts/ && \
cp -r /tmp/AdminApiScripts/AdminApi/Artifacts/PgSql/Structure/Admin/. /tmp/AdminApiScripts/PgSql/ && \
rm -f /tmp/EdFi_AdminApi_Scripts.zip && \
Expand Down
2 changes: 1 addition & 1 deletion Docker/dbadmin.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ ENV POSTGRES_USER=${POSTGRES_USER}
ENV POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
ENV POSTGRES_DB=postgres

COPY Settings/DB-Admin/pgsql/run-adminapi-migrations.sh /docker-entrypoint-initdb.d/3-run-adminapi-migrations.sh
COPY Application/EdFi.Ods.AdminApi/Artifacts/PgSql/Structure/Admin/ /tmp/AdminApiScripts/PgSql
COPY Settings/DB-Admin/pgsql/run-adminapi-migrations.sh /docker-entrypoint-initdb.d/3-run-adminapi-migrations.sh

RUN apk --no-cache add dos2unix=~7.4 unzip=~6.0 && \
dos2unix /docker-entrypoint-initdb.d/3-run-adminapi-migrations.sh && \
Expand Down
7 changes: 3 additions & 4 deletions Docker/dev.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,12 @@ LABEL maintainer="Ed-Fi Alliance, LLC and Contributors <techsupport@ed-fi.org>"
ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false
ENV ASPNETCORE_ENVIRONMENT Production

WORKDIR /app
COPY --from=publish /app/EdFi.Ods.AdminApi .

COPY Settings/dev/run.sh /app/run.sh
COPY Settings/dev/log4net.config /app/log4net.txt

WORKDIR /app
COPY --from=publish /app/EdFi.Ods.AdminApi .

RUN apk --no-cache add curl=~8 dos2unix=~7 bash=~5 gettext=~0 icu=~72 && \
cp /app/log4net.txt /app/log4net.config && \
dos2unix /app/*.json && \
Expand All @@ -42,6 +42,5 @@ RUN apk --no-cache add curl=~8 dos2unix=~7 bash=~5 gettext=~0 icu=~72 && \
chmod 700 /app/*.sh -- **

EXPOSE 443
WORKDIR /app

ENTRYPOINT ["/app/run.sh"]
36 changes: 0 additions & 36 deletions Docker/pgsql.Dockerfile

This file was deleted.