Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Workflow and functionalities to provide DFG-Viewer as a Docker image and use it #323

Open
wants to merge 52 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
7dc1f27
initial commit of docker and publish workflow
markusweigelt Oct 14, 2024
c50980e
Add extension repositories to workflow
markusweigelt Oct 14, 2024
d6db503
Update workflow and env
markusweigelt Oct 14, 2024
ab6bc34
Use dev-main branch for docker-image
markusweigelt Oct 14, 2024
633630a
Add sites configuration
markusweigelt Oct 14, 2024
0d516ac
Update site configuration
markusweigelt Oct 14, 2024
5c56156
Improve site configuration
markusweigelt Oct 14, 2024
19db23e
Rename workflow and change image name
markusweigelt Oct 15, 2024
f4e418b
Update to TYPO3 12
markusweigelt Nov 7, 2024
819a9a9
Update php version
markusweigelt Nov 7, 2024
81c2743
Update publish workflow
markusweigelt Nov 7, 2024
8c33bdc
Updates regarding building docker image
markusweigelt Nov 8, 2024
cc8d040
Improvements
markusweigelt Nov 8, 2024
205e74f
Add missing build args, ignore .env
markusweigelt Nov 8, 2024
a7890f6
Update workflow and gitignore
markusweigelt Nov 12, 2024
de7fa4d
Update composer.json
markusweigelt Nov 12, 2024
683fb87
Update job name
markusweigelt Nov 12, 2024
080d025
Replace required version
markusweigelt Nov 12, 2024
66079d6
Remove unused actions
markusweigelt Nov 12, 2024
eb03fec
echo cat
markusweigelt Nov 12, 2024
6d42cf3
Update publish action
markusweigelt Nov 12, 2024
e71f681
Update publish
markusweigelt Nov 12, 2024
7ad8c53
Add minimum stability
markusweigelt Nov 12, 2024
eac31f4
Add Dockerfile and improve workflow
markusweigelt Nov 12, 2024
c401b56
Update workflow
markusweigelt Nov 12, 2024
a76101b
Improvements
markusweigelt Nov 12, 2024
4960b58
Add image test
markusweigelt Nov 13, 2024
e919ae9
Improve path for extension checkout
markusweigelt Nov 13, 2024
c46ea23
Reset to default working directory
markusweigelt Nov 13, 2024
74e77aa
Reset to default working directory
markusweigelt Nov 13, 2024
2140e21
Update test of image and rename workflow
markusweigelt Nov 13, 2024
a160e04
Update path
markusweigelt Nov 13, 2024
f0604f5
Fix path
markusweigelt Nov 13, 2024
dc9c12f
Improve deploy image to GitHub Container Repository
markusweigelt Nov 13, 2024
16c5c08
Update doc
markusweigelt Nov 13, 2024
ab8a257
Using local image for tests
markusweigelt Nov 13, 2024
e45f252
Overwrite image path
markusweigelt Nov 13, 2024
715d665
Improve env path
markusweigelt Nov 13, 2024
a1fdcf7
Show env file
markusweigelt Nov 13, 2024
29248d1
Upate publish workflow
markusweigelt Nov 13, 2024
c001294
Add documentation of 3d viewer installation, diverse improvements
markusweigelt Nov 18, 2024
e0e3f95
Update documentation
markusweigelt Nov 18, 2024
f50bbd0
Remove unused overwrite and improve documentation
markusweigelt Nov 18, 2024
499e7b7
Update doc
markusweigelt Nov 18, 2024
361151c
Update Docker/build/Dockerfile
markusweigelt Nov 19, 2024
cbf4ff5
Update Dockerfile
markusweigelt Nov 19, 2024
fbbbea6
Fix codacy yaml
markusweigelt Nov 21, 2024
52f93aa
Improve syntax of doc
markusweigelt Nov 21, 2024
9d85bb9
Fix codacy issues
markusweigelt Nov 21, 2024
6c0c6c9
Improve doc for installing 3d viewer
markusweigelt Nov 21, 2024
1ce6923
Update Dockerfile
markusweigelt Nov 25, 2024
3e0883d
add env to configure trusted host pattern
markusweigelt Nov 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .github/scripts/test-image.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/bin/bash
cd ./Docker || exit 1

docker compose up -d

# run wait for it
chmod +x ./build/wait-for-it.sh
./build/wait-for-it.sh localhost:80 --strict -- echo "Application container is up"

echo "Wait 60 seconds until the installation process has finished"
sleep 60

# testing the container
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" localhost);
if [ "$HTTP_CODE" -ne 200 ]; then
echo "Expected HTTP status code to be 200, but got $HTTP_CODE"
exit 1
fi

HTML_TITLE=$(curl localhost | grep -m 1 -oP '(?<=<title>).+?(?=</title>)');
if [ "$HTML_TITLE" != "DFG Viewer" ]; then
echo "Expected content of the HTML <title> tag should to be \"DFG Viewer\", but got \"$HTML_TITLE\""
exit 1
fi
111 changes: 111 additions & 0 deletions .github/workflows/build-test-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
name: Build, test and publish DFG-Viewer image

on:
push:
branches: [ "master" ]
workflow_dispatch: # run manually

jobs:
build-test-publish:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4

# Deprecated should be set to the main repository if the update-typo12 branch is merged.
- name: Checkout DFG-Viewer repository
uses: actions/checkout@v4
with:
repository: markusweigelt/dfg-viewer
ref: update-typo12
path: ./Docker/build/extensions/dfg-viewer

- name: Checkout Kitodo.Presentation repository
uses: actions/checkout@v4
with:
repository: kitodo/kitodo-presentation
path: ./Docker/extensions/kitodo-presentation

# Deprecated should be set to the main repository if the update-typo12 branch is merged.
- name: Checkout SLUB Digital Collections repository
uses: actions/checkout@v4
with:
repository: markusweigelt/slub_digitalcollections
ref: update-typo12
path: ./Docker/build/extensions/slub_digitalcollections

- name: Prepare environment
run: |
# Rename example .env file
mv ./Docker/.env.example ./Docker/.env

# overwrite image path
sed -i "s|ghcr.io/slub/dfg-viewer|ghcr.io/${{ github.repository }}|g" ./Docker/.env

# Replace required versions with @dev
cd ./Docker/build/extensions/
echo "`jq '.require."kitodo/presentation"="@dev"' dfg-viewer/composer.json`" > dfg-viewer/composer.json
echo "`jq '.require."slub/slub-digitalcollections"="@dev"' dfg-viewer/composer.json`" > dfg-viewer/composer.json
echo "`jq '.require."kitodo/presentation"="@dev"' slub_digitalcollections/composer.json`" > slub_digitalcollections/composer.json

- name: Determine build args
run: |
date -u +"build_date=%Y-%m-%dT%H:%M:%SZ" >> $GITHUB_ENV
echo "vcs_ref=`git -C ./Docker/build/extensions/dfg-viewer rev-parse HEAD`" >> $GITHUB_ENV

# Activate cache export feature to reduce build time of images
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Build the DFG-Viewer image and cache to GitHub Actions cache
uses: docker/build-push-action@v6
with:
context: ./Docker/build
tags: ghcr.io/${{ github.repository }}:latest
build-args: |
BUILD_DATE=${{ env.build_date }}
VCS_REF=${{ env.vcs_ref }}
VCS_URL=https://github.com/${{ github.repository }}/tree/${{ github.ref_name }}/
cache-to: type=gha,mode=max,scope=${{ env.vcs_ref }}-image

- name: Test the DFG-Viewer image
run: |
chmod +x ./.github/scripts/test-image.sh
./.github/scripts/test-image.sh

- name: Collect docker logs on failure
if: failure()
uses: jwalton/gh-docker-logs@v2
with:
dest: './docker-logs'

- name: Tar docker logs on failure
if: failure()
run: tar cvzf ./docker-logs.tgz ./docker-logs

- name: Upload docker logs to GitHub
if: failure()
uses: actions/upload-artifact@v4
with:
name: docker-logs.tgz
path: ./docker-logs.tgz

- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Build the DFG-Viewer image using GitHub Actions cache and deploy to GitHub Container Repository
uses: docker/build-push-action@v6
with:
context: ./Docker/build
push: true
tags: ghcr.io/${{ github.repository }}:latest
build-args: |
BUILD_DATE=${{ env.build_date }}
VCS_REF=${{ env.vcs_ref }}
VCS_URL=https://github.com/${{ github.repository }}/tree/${{ github.ref_name }}/
cache-from: type=gha,scope=${{ env.vcs_ref }}-image

3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,6 @@ Documentation-GENERATED-temp/
public/
vendor/
composer.lock
.env
Docker/build/extensions
!Docker/build/extensions/.gitkeep
17 changes: 17 additions & 0 deletions Docker/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# name identifying the compose project
COMPOSE_PROJECT_NAME=dfg-viewer

# application image
APP_IMAGE=ghcr.io/slub/dfg-viewer:latest
APP_PORT=80
APP_T3_USER=admin
APP_T3_PASSWORD="dvAdmin.1"
APP_T3_CONFIG_SYS_TRUSTEDHOSTSPATTERN=""

# database image
DB_IMAGE=mysql:8.0
DB_PORT=3306
DB_HOST=db
DB_NAME=typo3
DB_USER=typo3
DB_PASSWORD=password
161 changes: 161 additions & 0 deletions Docker/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
# DFG Viewer Docker

* [Prerequisites](#prerequisites)
* [Usage](#usage)
* [Development](#development)
* [Further information](#further-information)

## Prerequisites

Install Docker Engine
<https://docs.docker.com/get-docker/>

Install Docker Compose
<https://docs.docker.com/compose/install/>

## Usage

Go to the directory where you've put `docker-compose.yml`.

Copy the environment file `.env.example` inside the directory and rename it to `.env`. Adjust the configuration of the respective service to suit your needs. The variables are marked with the prefix of the service e.g. `APP_` for our TYPO3 application with DFG-Viewer.

*It is recommended to adjust the password of the TYPO3 admin user `APP_T3_PASSWORD`, the database password `DB_PASSWORD` in general, especially in a productive environment.*

Download images and start all service containers

```bash
docker compose up -d
```

It may take about a minute for TYPO3 and the extension to be installed at the initial start.

The DFG Viewer instance can then be accessed under `localhost`.

*When running `docker compose up` all services e.g. DFG-Viewer (APP) and database (DB) in our `docker-compose.yml` will be started and each as separate Docker container.*

Stops all service containers

```bash
docker compose stop
```

Stops and remove all service containers

```bash
docker compose down
```

## Development

To build the image, a `build` folder with a subfolder `extension` must be added.

```bash
mkdir -p build/extensions
```

Then, the three extension repositories, [DFG Viewer](https://github.com/slub/dfg-viewer), [Kitodo.Presentation](https://github.com/kitodo/kitodo-presentation), [SLUB Digital Collections](https://github.com/slub/slub_digitalcollections), must be cloned into this folder.

```bash
git clone https://github.com/slub/dfg-viewer build/extensions
git clone https://github.com/kitodo/kitodo-presentation build/extensions
git clone https://github.com/slub/slub_digitalcollections build/extensions
```

Adjust the `composer.json` files of the checkouts as follows.

`composer.json` of the [DFG Viewer](https://github.com/slub/dfg-viewer) checkout `build/extensions/dfg-viewer`:

```text
...
"require": {
...
"kitodo/presentation": "@dev"
"slub/slub-digitalcollection": "@dev"
...
},
...
```

`composer.json` of the [SLUB Digital Collections](https://github.com/slub/slub_digitalcollections) checkout `build/extensions/slub_digitalcollections`:

```text
...
"require": {
...
"kitodo/presentation": "@dev"
...
},
...
```

Build the image.

*Ensure that the `.env` file has been created. It is recommended to adjust the password of the TYPO3 admin user `APP_T3_PASSWORD`, the database password `DB_PASSWORD` in general and the `APP_IMAGE` name for building custom images in `.env` file.*

```bash
docker compose build dfg-viewer-app
```

## Further information

### Usage of 3D viewer integrations

*There are multiple ways to install the 3D viewer integration. The simplest is to upload the folder that is generated during the [viewer integration installation](https://github.com/slub/dlf-3d-viewers#installation). We have decided to provide an installation based on commands here.*

Once all containers have been started (see [Usage](#usage)), you can run following command:

```bash
docker exec -u root -i dfg-viewer-app-1 sh -c "\
apt-get update && apt-get install unzip \
&& wget "https://github.com/slub/dlf-3d-viewers/archive/refs/heads/main.zip" -O /tmp/dlf-3d-viewers.zip \
&& cd /var/www/html/dfg-viewer/public/fileadmin\
&& unzip /tmp/dlf-3d-viewers.zip\
&& mv dlf-3d-viewers-main dlf_3d_viewers\
&& rm /tmp/dlf-3d-viewers.zip\
&& sh dlf_3d_viewers/install.sh\
&& chown -R www-data:www-data dlf_3d_viewers"
```

The steps of this long command can also be executed individually:

1. Login into `dfg-viewer-app-1` container as root user.

```bash
docker exec -u root -it dfg-viewer-app-1 bash
```

2. Install unzip command line tool.

```bash
apt-get update && apt-get install unzip
```

3. Download the current state of main branch of repository [3D viewer integrations for DFG-Viewer](https://github.com/slub/dlf-3d-viewers) as zip file

```bash
wget "https://github.com/slub/dlf-3d-viewers/archive/refs/heads/main.zip" -O /tmp/dlf-3d-viewers.zip
```

4. Navigate to `fileadmin` folder, unzip the zip file, rename unzipped folder to `dlf_3d_viewers` and remove zip file

```bash
cd /var/www/html/dfg-viewer/public/fileadmin

unzip /tmp/dlf-3d-viewers.zip

mv dlf-3d-viewers-main dlf_3d_viewers

rm /tmp/dlf-3d-viewers.zip
```

5. Download libraries and frameworks for each viewer integration

```bash
sh dlf_3d_viewers/install.sh
```

6. Change owner of integration folder, subfolder and files

```bash
chown -R www-data:www-data dlf_3d_viewers
```
Loading