Configuring Jenkins CI

There is a Jenkinsfile included in this holodeck-engine repo for use with Jenkins Blue Ocean. Our CI setup makes use of the following:

• Docker
• The jenkinsci/blueocean docker image
• ue4-docker
• ue4cli

Jenkins is run inside a docker container, and Holodeck itself is built inside a docker container generated by ue4-docker. ue4-cli is used to make the build process easier.

Note: Jenkins Docker Image

We must use a custom docker image, since we need git-lfs support. This is explained below.

Configuring a new build server

Install Docker and Python 3.6+, you'll need these later.

Step 1: Build ue4-docker image

You'll want to do this first, since it takes several hours.

1. Follow the instructions for Configuring Linux

   You might need to add ppas for Python 3.6 (-dev too!) and then python3.6 -m pip install ... to install the pip package

2. Build the ue4-complete image. Make sure to specify the current Holodeck engine version.

   Note that you will need ~300gb of disk space for this to succeed! The requirements aren't overestimating

Step 2: Install Jenkins

We run Jenkins in a docker container, and use a Docker volume to persist data (instead of a bind mount. Avoids permissions issues).

1. Install jenkinsci/blueocean

   docker pull jenkinsci/blueocean

2. Create our custom docker image, with Git LFS

   In a new directory, create a file named Dockerfile that contains the following:

   FROM jenkinsci/blueocean
   USER root
   RUN apk add git-lfs
   

   Then, run docker build -t holodeck/blueocean-lfs .

3. Create a docker volume

   docker volume create jenkins-data

4. Copy this sample startup script somewhere

   docker run \
    -u root \
    --rm \
    -d \
    -p 80:8080 \
    -p 50000:50000 \
    -v jenkins-data:/var/jenkins_home \
    -v /var/run/docker.sock:/var/run/docker.sock \
    holodeck/blueocean-lfs

   • -u runs Jenkins as the root user in the container

   • --rm cleans up any temporary volumes

   • -p 80:8080 forwards port 8080 in the container to port 80 on the host

   • -v jenkins-data.. binds /var/jenkins_home in the container to the docker volume jenkins-data

   • -v -v /var/run/docker.sock... allows the Jenkins instance inside the container to access the docker daemon outside of the container so that the dockerized Jenkins can have docker containers for builds.

     (we heard you liked docker so we put docker inside your docker so you can docker while you docker)

5. Configure this script to be run on startup with a systemd unit file

   In /etc/systemd/system/jenkins.service, place this file:

   [Unit]
   Description=Jenkins in a Docker Image for Holodeck
   After=docker.service
   Requires=docker.service
   
   [Service]
   TimeoutStartSec=0
   Restart=always
   # User=remote
   ExecStart=/home/remote/holodeck/jenkins/start-jenkins.sh
   
   [Install]
   WantedBy=multi-user.target

6. Start the Jenkins systemd unit by running sudo systemctl start jenkins.service. You can check the logs with sudo journalctl -u jenkins.service or the status with sudo systemctl status jenkins.service.

Step 3: Configure Jenkins

1. Connect to localhost, wait until you get the Unlock Jenkins screen.

   We will need to put in a password that Jenkins generated. You can do this by inspecting the systemd log (sudo journalctl -u jenkins.service) or by entering the docker container entering the docker container.

2. Run docker ps - note the name for the jenkinsci/blueocean container (eg vigilant_mayer)

3. Run docker exec -it vigilant_mayer bash - this should get you a shell inside the container. Get the password from /var/jenkins_home/secrets/initialAdminPassword

4. Install suggested plugins, create an admin user, wait for it to restart.

5. Navigate to localhost/blue, you should see the new-looking Jenkins Blue UI instead of the janky old one

6. Select "Create new Pipeline" -> "Github"

7. Follow the link to create a new Personal Access Token. Write it down somewhere, once generated you can never see it again!

   Note - I think this token is associated with a user and not the organization, so if my GitHub account (jaydenmilne) gets removed from the org, Jenkins might break.

8. Select the Holodeck-Engine repo, it should detect the repo and begin to build it

9. Configure Jenkins to periodically poll the holodeck-engine repo. a. From the Jenkins dashboard, select holodeck-engine b. Select "Configure" on the left hand side c. Select the "Scan Repository Triggers" tab d. Check "Periodically if not otherwise run" and choose a short interval, like 5 minutes or so.

   If it doesn't there could be an issue with the Jenkins server being publicly accessible, if it isn't on the open internet, Github won't be able to use webhooks to notify Jenkins of a new commit or PR. You will have to manually make Jenkins scan the repo every time:

   • Open the configuration for the pipeline
   • "Scan Repository Now" on the left

   If the server is publicly available and builds still aren't automatically happening, you may be a victim of JENKINS-50883. See this blog post