From d35eef8ef3862231589a1d099e57ea202746f908 Mon Sep 17 00:00:00 2001 From: Javier Morales Date: Mon, 18 Jul 2022 16:48:58 -0400 Subject: [PATCH] Split development into subsections and removed some old tutorials --- docs/development.md | 318 ---------------------------------- docs/development/README.md | 21 +++ docs/development/debugging.md | 35 ++++ docs/development/docs.md | 23 +++ docs/development/release.md | 19 ++ docs/development/scripts.md | 49 ++++++ docs/development/windows.md | 31 ++++ 7 files changed, 178 insertions(+), 318 deletions(-) delete mode 100644 docs/development.md create mode 100644 docs/development/README.md create mode 100644 docs/development/debugging.md create mode 100644 docs/development/docs.md create mode 100644 docs/development/release.md create mode 100644 docs/development/scripts.md create mode 100644 docs/development/windows.md diff --git a/docs/development.md b/docs/development.md deleted file mode 100644 index c50c79bf..00000000 --- a/docs/development.md +++ /dev/null @@ -1,318 +0,0 @@ ---- -layout: default -title: Development -nav_order: 14 ---- - -# Development - -This document contains development notes and tips for working with Thundernetes source code. - -## Release new Thundernetes version - -This will require 2 PRs. - -- Make sure you update `.versions` file on the root of this repository with the new version -- Run `make clean` to ensure any cached artifacts of old builds are deleted. -- Push and merge -- Manually run the GitHub Actions workflows to create new [linux images](https://github.com/PlayFab/thundernetes/actions/workflows/publish.yml) and [windows images](https://github.com/PlayFab/thundernetes/actions/workflows/publish-windows.yml) -- Git pull the latest changes from the main branch -- Run `make create-install-files` to generate the operator install files -- Replace the image on the [netcore-sample YAML files](https://github.com/PlayFab/thundernetes/samples/netcore) -- Push and merge - -## Metrics - -- If you are using Prometheus and Prometheus operator, uncomment all sections with `# [PROMETHEUS]` on `config/default/kustomization.yaml` file. More details [here](https://book.kubebuilder.io/reference/metrics.html) -- To enable authentication for the metrics server, remove the comment from this line on the file ``config/default/kustomization.yaml`: `- manager_auth_proxy_patch.yaml` - -## Running end to end tests on macOS - -First of all, end to end tests require `envsubst` utility, assuming that you have Homebrew installed you can get it via `brew install gettext && brew link --force gettext`. -We assume that you have installed Go, then you should install kind with `go install sigs.k8s.io/kind@latest`. Kind will be installed in `$(go env GOPATH)/bin` directory. Then, you should move kind to the `/operator/testbin/bin/` folder with a command like `cp $(go env GOPATH)/bin/kind ./operator/testbin/bin/kind`. You can run end to end tests with `make builddockerlocal createkindcluster e2elocal`. - -## Various scripts - -### Generate cert for testing - -``` -openssl genrsa 2048 > private.pem -openssl req -x509 -days 1000 -new -key private.pem -out public.pem -kubectl create namespace thundernetes-system -kubectl create secret tls tls-secret -n thundernetes-system --cert=public.pem --key=private.pem -``` - -### Allocate a game server - -#### With TLS auth - -```bash -IP=$(kubectl get svc -n thundernetes-system thundernetes-controller-manager -o jsonpath='{.status.loadBalancer.ingress[0].ip}') -curl --key ~/private.pem --cert ~/public.pem --insecure -H 'Content-Type: application/json' -d '{"buildID":"85ffe8da-c82f-4035-86c5-9d2b5f42d6f5","sessionID":"85ffe8da-c82f-4035-86c5-9d2b5f42d6f5"}' https://${IP}:5000/api/v1/allocate -``` - -#### Without TLS auth - -```bash -IP=$(kubectl get svc -n thundernetes-system thundernetes-controller-manager -o jsonpath='{.status.loadBalancer.ingress[0].ip}') -curl -H 'Content-Type: application/json' -d '{"buildID":"85ffe8da-c82f-4035-86c5-9d2b5f42d6f5","sessionID":"85ffe8da-c82f-4035-86c5-9d2b5f42d6f5"}' http://${IP}:5000/api/v1/allocate -``` - -### Do 50 allocations - -#### Without TLS auth - -```bash -IP=$(kubectl get svc -n thundernetes-system thundernetes-controller-manager -o jsonpath='{.status.loadBalancer.ingress[0].ip}') -for i in {1..50}; do SESSION_ID=$(uuidgen); curl -H 'Content-Type: application/json' -d '{"buildID":"85ffe8da-c82f-4035-86c5-9d2b5f42d6f6","sessionID":"'${SESSION_ID}'"}' http://${IP}:5000/api/v1/allocate; done -``` - -#### With TLS auth - -```bash -IP=$(kubectl get svc -n thundernetes-system thundernetes-controller-manager -o jsonpath='{.status.loadBalancer.ingress[0].ip}') -for i in {1..50}; do SESSION_ID=$(uuidgen); curl --key ~/private.pem --cert ~/public.pem --insecure -H 'Content-Type: application/json' -d '{"buildID":"85ffe8da-c82f-4035-86c5-9d2b5f42d6f6","sessionID":"'${SESSION_ID}'"}' https://${IP}:5000/api/v1/allocate; done -``` - -## Run end to end tests locally - -```bash -make clean deletekindcluster builddockerlocal createkindcluster e2elocal -``` - -## Run controller locally - -```bash -cd operator -THUNDERNETES_INIT_CONTAINER_IMAGE=ghcr.io/playfab/thundernetes-initcontainer:0.2.0 go run main.go -``` - -## [ADVANCED] Install thundernetes via cloning this repository - -You should `git clone` this repository to your local machine. As soon as this is done, you can run the following command to install Thundernetes. - -```bash -export TAG=0.0.2.0 -IMG=ghcr.io/playfab/thundernetes-operator:${TAG} \ - IMAGE_NAME_INIT_CONTAINER=ghcr.io/playfab/thundernetes-initcontainer \ - IMAGE_NAME_SIDECAR=ghcr.io/playfab/thundernetes-sidecar-netcore \ - API_SERVICE_SECURITY=none \ - make -C pkg/operator install deploy -``` - -Note that this will install thundernetes without any security for the allocation API service. If you want to enable security for the allocation API service, you can should provide a certificate and key for the allocation API service. - -You can use OpenSSL to create a self-signed certificate and key (not recommended for production). - -```bash -openssl genrsa 2048 > private.pem -openssl req -x509 -days 1000 -new -key private.pem -out public.pem -``` - -Install the cert and key as a Kubernetes Secret in the same namespace as the operator. - -```bash -kubectl create namespace thundernetes-system -kubectl create secret tls tls-secret -n thundernetes-system --cert=public.pem --key=private.pem -``` - -Then, you need to install the operator enabling TLS authentication for the allocation API service. - -```bash -export TAG=0.3.0 -IMG=ghcr.io/playfab/thundernetes-operator:${TAG} \ - IMAGE_NAME_INIT_CONTAINER=docker.io/dgkanatsios/thundernetes-initcontainer \ - IMAGE_NAME_SIDECAR=docker.io/dgkanatsios/thundernetes-sidecar-netcore \ - API_SERVICE_SECURITY=usetls \ - make -C pkg/operator install deploy -``` - -As soon as this is done, you can run `kubectl -n thundernetes-system get pods` to verify that the operator pod is running. To run a demo gameserver, you can use the command: - -```bash -kubectl apply -f pkg/operator/config/samples/netcore.yaml -``` - -This will create a GameServerBuild with 2 standingBy and 4 maximum gameservers. -After a while, you will see your game servers. - -```bash -kubectl get gameservers # or kubectl get gs -``` - -```bash -NAME HEALTH STATE PUBLICIP PORTS SESSIONID -gameserverbuild-sample-apbjz Healthy StandingBy 52.183.89.4 80:24558 -gameserverbuild-sample-gqhrm Healthy StandingBy 52.183.88.255 80:10319 -``` - -and your GameServerBuild: - -```bash -kubectl get gameserverbuild # or kubectl get gsb -``` - -```bash -NAME ACTIVE STANDBY CRASHES HEALTH -gameserverbuild-sample 0 2 0 Healthy -``` - -You can edit the number of standingBy and max by changing the values in the GameServerBuild. - -```bash -kubectl edit gsb gameserverbuild-sample -``` - -You can also use the allocation API. To get the Public IP, you can use the command: - -```bash -kubectl get svc -n thundernetes-system thundernetes-controller-manager -``` - -```bash -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -thundernetes-controller-manager LoadBalancer 10.0.62.144 20.83.72.255 5000:32371/TCP 39m -``` - -The External-Ip field is the Public IP of the LoadBalancer that we can use to call the allocation API. - -If you have configured your allocation API service with no security: - -```bash -IP=... -curl -H 'Content-Type: application/json' -d '{"buildID":"85ffe8da-c82f-4035-86c5-9d2b5f42d6f5","sessionID":"85ffe8da-c82f-4035-86c5-9d2b5f42d6f5"}' http://${IP}:5000/api/v1/allocate -``` - -If you're using TLS authentication: - -```bash -IP=... -curl --key ~/private.pem --cert ~/public.pem --insecure -H 'Content-Type: application/json' -d '{"buildID":"85ffe8da-c82f-4035-86c5-9d2b5f42d6f5","sessionID":"85ffe8da-c82f-4035-86c5-9d2b5f42d6f5"}' https://${IP}:5000/api/v1/allocate -``` - -Then, you can see that the game server has successfully been allocated. - -```bash -kubectl get gameservers -``` - -```bash -NAME HEALTH STATE PUBLICIP PORTS SESSIONID -gameserverbuild-sample-apbjz Healthy StandingBy 52.183.89.4 80:24558 -gameserverbuild-sample-bmich Healthy Active 20.94.219.110 80:38208 85ffe8da-c82f-4035-86c5-9d2b5f42d6f5 -gameserverbuild-sample-gqhrm Healthy StandingBy 52.183.88.255 80:10319 -``` - -On the allocated server, you can call the demo game server HTTP endpoint. - -```bash -curl 20.94.219.110:38208/hello -``` - -You'll get a response like `Hello from `. - -Try calling the endpoint that will gracefully terminate the game server. - -```bash -curl 20.94.219.110:38208/hello/terminate -``` - -We can see that the game server has been terminated. Since our GameServerBuild is configured with 2 standingBys, no other servers are created. - -```bash -kubectl get gameservers -``` - -```bash -NAME HEALTH STATE PUBLICIP PORTS SESSIONID -gameserverbuild-sample-apbjz Healthy StandingBy 52.183.89.4 80:24558 -gameserverbuild-sample-gqhrm Healthy StandingBy 52.183.88.255 80:10319 -``` - -## kubebuilder notes - -Project was bootstrapped using [kubebuilder](https://github.com/kubernetes-sigs/kubebuilder) using the following commands: - -```bash -kubebuilder init --domain playfab.com --repo github.com/playfab/thundernetes/pkg/operator -kubebuilder create api --group mps --version v1alpha1 --kind GameServer -kubebuilder create api --group mps --version v1alpha1 --plural gameserverbuilds --kind GameServerBuild -kubebuilder create api --group mps --version v1alpha1 --plural gameserverdetails --kind GameServerDetail -``` - -## env variables sample - -```bash -PUBLIC_IPV4_ADDRESS=20.184.250.154 -PF_REGION=WestUs -PF_VM_ID=xcloudwus4u4yz5dlozul:WestUs:6b5973a5-a3a5-431a-8378-eff819dc0c25:tvmps_efa402aacd4f682230cfd91bd3dc0ddfae68c312f2b6905577cb7d9424681930_d -PF_SHARED_CONTENT_FOLDER=/gsdkdata/GameSharedContent -PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin -PF_SERVER_INSTANCE_NUMBER=2 -PWD=/app -PF_BUILD_ID=88a958b9-14fb-4ad9-85ca-5cc13207232e -GSDK_CONFIG_FILE=/gsdkata/Config/gsdkConfig.json -SHLVL=1 -HOME=/root -CERTIFICATE_FOLDER=/gsdkdata/GameCertificates -PF_SERVER_LOG_DIRECTORY=/gsdkdata/GameLogs/ -PF_TITLE_ID=1E03 -_=/usr/bin/env -``` - -## Docker compose - -The docker-compose.yml file on the root of this repo was created to facilitate sidecar development. - -## Test your changes to a cluster - -To test your changes to thundernetes to a Kubernetes cluster, you can use the following steps: - -- The Makefile on the root of the project contains a variable `NS` that points to the container registry that you use during development. So you'd need to either set the variable in your environment (`export NS=`) or set it before calling `make` (like `NS= make build push`). -- Login to your container registry (`docker login`) -- Run `make clean build push` to build the container images and push them to your container registry -- Run `create-install-files-dev` to create the install files for the cluster -- Checkout the `installfilesdev` folder for the generated install files. This file is included in .gitignore so it will never be committed. -- Test your changes as required. -- single command: `NS=docker.io// make clean build push create-install-files-dev` - -## Testing with Windows containers - -Thundernetes now supports game servers running on Windows containers, you can read more about it [here](./howtos/windowscontainers.md). If you want to try this you need both a Windows machine, with Windows 2019 or higher, to build the necessary Windows Docker images, and a Kubernetes cluster with Windows nodes. You can't build or run Windows containers on a Linux machine. If you have all of this, you can follow these next steps: - -- Login to your container registry (`docker login `) on your Linux machine or WSL, where `` is the registry where you want to upload your images. -- Run `NS= make clean build push create-install-files-dev`. -- Login to your container registry (`docker login`) on your Windows machine. -- Run `.\windows\Build-DockerWin.ps1 -registry `. -- Now you can install Thundernetes on your cluster using any of the files on the `installfilesdev` directory. -- If you want to deploy a Windows game server on Thundernetes make sure to include the following on the game server build YAML file, we use this to know how to deploy the game servers correctly: - -```yaml -apiVersion: mps.playfab.com/v1alpha1 -kind: GameServerBuild -metadata: - name: gameserverbuild-sample -spec: - ... - template: - spec: - nodeSelector: - kubernetes.io/os: windows - ... -``` -## Running docs locally - -We use [GitHub Pages](https://docs.github.com/en/pages) to host Thundernetes documentation. To preview your changes locally: - -- [WSL](https://docs.microsoft.com/en-us/windows/wsl/install) is recommended -- Follow the instructions [here](https://docs.github.com/en/pages/setting-up-a-github-pages-site-with-jekyll/testing-your-github-pages-site-locally-with-jekyll). Specifically, install `Ruby` and `Bundler`. `Jekyll` needs to be installed as well but it's already in the Gemfile so it should be installed automatically. -- Switch to the `docs` directory -- Run `bundle install` to install the necessary prerequisites -- Run `bundle exec jekyll serve --config _config-development.yml` -- Browse the site on http://localhost:4000/thundernetes - -Alternatively, you can use [this container image](https://github.com/BretFisher/jekyll-serve) with a command similar to the following, once you are in the `docs` directory: - -```bash -docker run -p 4000:4000 --env JEKYLL_ENV=production --rm -v $(pwd):/site bretfisher/jekyll-serve bundle exec jekyll serve --force-polling --config _config-development.yml --host 0.0.0.0 -``` \ No newline at end of file diff --git a/docs/development/README.md b/docs/development/README.md new file mode 100644 index 00000000..252fdf82 --- /dev/null +++ b/docs/development/README.md @@ -0,0 +1,21 @@ +--- +layout: default +title: Development +nav_order: 14 +has_children: true +--- + +# Development + +This section contains development notes and tips for working with Thundernetes source code. + +## Kubebuilder notes + +Project was bootstrapped using [kubebuilder](https://github.com/kubernetes-sigs/kubebuilder) using the following commands: + +```bash +kubebuilder init --domain playfab.com --repo github.com/playfab/thundernetes/pkg/operator +kubebuilder create api --group mps --version v1alpha1 --kind GameServer +kubebuilder create api --group mps --version v1alpha1 --plural gameserverbuilds --kind GameServerBuild +kubebuilder create api --group mps --version v1alpha1 --plural gameserverdetails --kind GameServerDetail +``` diff --git a/docs/development/debugging.md b/docs/development/debugging.md new file mode 100644 index 00000000..22c05685 --- /dev/null +++ b/docs/development/debugging.md @@ -0,0 +1,35 @@ +--- +layout: default +title: Debugging +parent: Development +nav_order: 1 +--- + +# Debugging + +To test your local code you have 2 options: you can run the code in a local kind cluster, or you can build local container images, upload them to a container registry, and then deploy them to a cluster. + +## Run end to end tests locally + +This command will run the e2e tests locally, and it won't delete the cluster after it's done, so you can either deploy more GameServerBuilds or check the ones used for the tests under the `e2e` namespace. + +```bash +make clean deletekindcluster builddockerlocal createkindcluster e2elocal +``` + +### Running end to end tests on macOS + +First of all, end to end tests require `envsubst` utility, assuming that you have Homebrew installed you can get it via `brew install gettext && brew link --force gettext`. +We assume that you have installed Go, then you should install kind with `go install sigs.k8s.io/kind@latest`. Kind will be installed in `$(go env GOPATH)/bin` directory. Then, you should move kind to the `/operator/testbin/bin/` folder with a command like `cp $(go env GOPATH)/bin/kind ./operator/testbin/bin/kind`. You can run end to end tests with `make clean builddockerlocal createkindcluster e2elocal`. + +## Test your changes on a cluster + +To test your changes to Thundernetes on a Kubernetes cluster, you can use the following steps: + +- The Makefile on the root of the project contains a variable `NS` that points to the container registry that you use during development. So you'd need to either set the variable in your environment (`export NS=`) or set it before calling `make` (like `NS= make build push`). +- Login to your container registry (`docker login`) +- Run `make clean build push` to build the container images and push them to your container registry +- Run `create-install-files-dev` to create the install files for the cluster +- Checkout the `installfilesdev` folder for the generated install files. This file is included in .gitignore so it will never be committed. +- Test your changes as required. +- single command: `NS=docker.io// make clean build push create-install-files-dev` diff --git a/docs/development/docs.md b/docs/development/docs.md new file mode 100644 index 00000000..a37f2fb5 --- /dev/null +++ b/docs/development/docs.md @@ -0,0 +1,23 @@ +--- +layout: default +title: Running docs locally +parent: Development +nav_order: 3 +--- + +# Running docs locally + +We use [GitHub Pages](https://docs.github.com/en/pages) to host Thundernetes documentation. To preview your changes locally: + +- [WSL](https://docs.microsoft.com/en-us/windows/wsl/install) is recommended +- Follow the instructions [here](https://docs.github.com/en/pages/setting-up-a-github-pages-site-with-jekyll/testing-your-github-pages-site-locally-with-jekyll). Specifically, install `Ruby` and `Bundler`. `Jekyll` needs to be installed as well but it's already in the Gemfile so it should be installed automatically. +- Switch to the `docs` directory +- Run `bundle install` to install the necessary prerequisites +- Run `bundle exec jekyll serve --config _config-development.yml` +- Browse the site on http://localhost:4000/thundernetes + +Alternatively, you can use [this container image](https://github.com/BretFisher/jekyll-serve) with a command similar to the following, once you are in the `docs` directory: + +```bash +docker run -p 4000:4000 --env JEKYLL_ENV=production --rm -v $(pwd):/site bretfisher/jekyll-serve bundle exec jekyll serve --force-polling --config _config-development.yml --host 0.0.0.0 +``` diff --git a/docs/development/release.md b/docs/development/release.md new file mode 100644 index 00000000..ceb33bb6 --- /dev/null +++ b/docs/development/release.md @@ -0,0 +1,19 @@ +--- +layout: default +title: Release new Thundernetes version +parent: Development +nav_order: 4 +--- + +# Release new Thundernetes version + +This will require 2 PRs. + +- Make sure you update `.versions` file on the root of this repository with the new version +- Run `make clean` to ensure any cached artifacts of old builds are deleted. +- Push and merge +- Manually run the GitHub Actions workflows to create new [linux images](https://github.com/PlayFab/thundernetes/actions/workflows/publish.yml) and [windows images](https://github.com/PlayFab/thundernetes/actions/workflows/publish-windows.yml) +- Git pull the latest changes from the main branch +- Run `make create-install-files` to generate the operator install files +- Replace the image on the [netcore-sample YAML files](https://github.com/PlayFab/thundernetes/samples/netcore) +- Push and merge \ No newline at end of file diff --git a/docs/development/scripts.md b/docs/development/scripts.md new file mode 100644 index 00000000..684e6da5 --- /dev/null +++ b/docs/development/scripts.md @@ -0,0 +1,49 @@ +--- +layout: default +title: Useful scripts +parent: Development +nav_order: 5 +--- + +# Useful scripts + +## Generate cert for testing + +``` +openssl genrsa 2048 > private.pem +openssl req -x509 -days 1000 -new -key private.pem -out public.pem +kubectl create namespace thundernetes-system +kubectl create secret tls tls-secret -n thundernetes-system --cert=public.pem --key=private.pem +``` + +## Allocate a game server + +### With TLS auth + +```bash +IP=$(kubectl get svc -n thundernetes-system thundernetes-controller-manager -o jsonpath='{.status.loadBalancer.ingress[0].ip}') +curl --key ~/private.pem --cert ~/public.pem --insecure -H 'Content-Type: application/json' -d '{"buildID":"85ffe8da-c82f-4035-86c5-9d2b5f42d6f5","sessionID":"85ffe8da-c82f-4035-86c5-9d2b5f42d6f5"}' https://${IP}:5000/api/v1/allocate +``` + +### Without TLS auth + +```bash +IP=$(kubectl get svc -n thundernetes-system thundernetes-controller-manager -o jsonpath='{.status.loadBalancer.ingress[0].ip}') +curl -H 'Content-Type: application/json' -d '{"buildID":"85ffe8da-c82f-4035-86c5-9d2b5f42d6f5","sessionID":"85ffe8da-c82f-4035-86c5-9d2b5f42d6f5"}' http://${IP}:5000/api/v1/allocate +``` + +## Do 50 allocations + +### Without TLS auth + +```bash +IP=$(kubectl get svc -n thundernetes-system thundernetes-controller-manager -o jsonpath='{.status.loadBalancer.ingress[0].ip}') +for i in {1..50}; do SESSION_ID=$(uuidgen); curl -H 'Content-Type: application/json' -d '{"buildID":"85ffe8da-c82f-4035-86c5-9d2b5f42d6f6","sessionID":"'${SESSION_ID}'"}' http://${IP}:5000/api/v1/allocate; done +``` + +### With TLS auth + +```bash +IP=$(kubectl get svc -n thundernetes-system thundernetes-controller-manager -o jsonpath='{.status.loadBalancer.ingress[0].ip}') +for i in {1..50}; do SESSION_ID=$(uuidgen); curl --key ~/private.pem --cert ~/public.pem --insecure -H 'Content-Type: application/json' -d '{"buildID":"85ffe8da-c82f-4035-86c5-9d2b5f42d6f6","sessionID":"'${SESSION_ID}'"}' https://${IP}:5000/api/v1/allocate; done +``` diff --git a/docs/development/windows.md b/docs/development/windows.md new file mode 100644 index 00000000..09654732 --- /dev/null +++ b/docs/development/windows.md @@ -0,0 +1,31 @@ +--- +layout: default +title: Developing with Windows containers +parent: Development +nav_order: 2 +--- + +# Developing with Windows containers + +Thundernetes now supports game servers running on Windows containers, you can read more about it [here](./howtos/windowscontainers.md). If you want to try this you need both a Windows machine, with Windows 2019 or higher, to build the necessary Windows Docker images, and a Kubernetes cluster with Windows nodes. You can't build or run Windows containers on a Linux machine. If you have all of this, you can follow these next steps: + +- Login to your container registry (`docker login `) on your Linux machine or WSL, where `` is the registry where you want to upload your images. +- Run `NS= make clean build push create-install-files-dev`. +- Login to your container registry (`docker login`) on your Windows machine. +- Run `.\windows\Build-DockerWin.ps1 -registry `. +- Now you can install Thundernetes on your cluster using any of the files on the `installfilesdev` directory. +- If you want to deploy a Windows game server on Thundernetes make sure to include the following on the game server build YAML file, we use this to know how to deploy the game servers correctly: + +```yaml +apiVersion: mps.playfab.com/v1alpha1 +kind: GameServerBuild +metadata: + name: gameserverbuild-sample +spec: + ... + template: + spec: + nodeSelector: + kubernetes.io/os: windows + ... +```