Skip to content

Commit

Permalink
Merge pull request #938 from ansible/kubernetes_install_support
Browse files Browse the repository at this point in the history
Kubernetes install support
  • Loading branch information
matburt authored Jan 10, 2018
2 parents 9365e47 + 7ea6d7b commit ae06cff
Show file tree
Hide file tree
Showing 7 changed files with 397 additions and 9 deletions.
93 changes: 86 additions & 7 deletions INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,24 +13,30 @@ This document provides a guide for installing AWX.
- [Choose a deployment platform](#choose-a-deployment-platform)
- [Official vs Building Images](#official-vs-building-images)
- [OpenShift](#openshift)
- [Prerequisites](#prerequisites)
- [Prerequisites](#prerequisites-1)
- [Deploying to Minishift](#deploying-to-minishift)
- [Pre-build steps](#pre-build-steps)
- [PostgreSQL](#postgresql)
- [Start the build](#start-the-build)
- [Post build](#post-build)
- [Accessing AWX](#accessing-awx)
- [Docker](#docker)
- [Kubernetes](#kubernetes)
- [Prerequisites](#prerequisites-2)
- [Pre-build steps](#pre-build-steps-1)
- [Start the build](#start-the-build-1)
- [Accessingm AWX](#accessing-awx-1)

This comment has been minimized.

Copy link
@philipsd6

philipsd6 Jan 25, 2018

Contributor

No! Don't add typos!!!

- [SSL Termination](#ssl-termination)
- [Docker](#docker)
- [Prerequisites](#prerequisites-3)
- [Pre-build steps](#pre-build-steps-2)
- [Deploying to a remote host](#deploying-to-a-remote-host)
- [Inventory variables](#inventory-variables)
- [Docker registry](#docker-registry)
- [PostgreSQL](#postgresql-1)
- [Proxy settings](#proxy-settings)
- [Start the build](#start-the-build-1)
- [Start the build](#start-the-build-2)
- [Post build](#post-build-1)
- [Accessing AWX](#accessing-awx-1)
- [Accessing AWX](#accessing-awx-2)

## Getting started

Expand Down Expand Up @@ -63,7 +69,7 @@ The system that runs the AWX service will need to satisfy the following requirem
- At leasts 4GB of memory
- At least 2 cpu cores
- At least 20GB of space
- Running Docker or Openshift
- Running Docker, Openshift, or Kubernetes

### AWX Tunables

Expand All @@ -75,7 +81,7 @@ We currently support running AWX as a containerized application using Docker ima

The [installer](./installer) directory contains an [inventory](./installer/inventory) file, and a playbook, [install.yml](./installer/install.yml). You'll begin by setting variables in the inventory file according to the platform you wish to use, and then you'll start the image build and deployment process by running the playbook.

In the sections below, you'll find deployment details and instructions for each platform. To deploy to Docker, view the [Docker section](#docker), and for OpenShift, view the [OpenShift section](#openshift).
In the sections below, you'll find deployment details and instructions for each platform. To deploy to Docker, view the [Docker section](#docker), for OpenShift, view the [OpenShift section](#openshift), for Kubernetes, view the [Kubernetes section](#kubernetes).

### Official vs Building Images

Expand Down Expand Up @@ -140,7 +146,7 @@ Before starting the build process, review the [inventory](./installer/inventory)
*docker_registry*

> IP address and port, or URL, for accessing a registry that the OpenShift cluster can access. Defaults to *172.30.1.1:5000*, the internal registry delivered with Minishift. This is not needed if you are using official hosted images.
n
*docker_registry_repository*

> Namespace to use when pushing and pulling images to and from the registry. Generally this will match the project name. It defaults to *awx*. This is not needed if you are using official hosted images.
Expand Down Expand Up @@ -267,6 +273,79 @@ The above example is taken from a Minishift instance. From a web browser, use `h

Once you access the AWX server, you will be prompted with a login dialog. The default administrator username is `admin`, and the password is `password`.

## Kubernetes

### Prerequisites

A Kubernetes deployment will require you to have access to a Kubernetes cluster as well as the following tools:

- [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
- [helm](https://docs.helm.sh/using_helm/#quickstart-guide)

The installation program will reference `kubectl` directly. `helm` is only necessary if you are letting the installer configure PostgreSQL for you.

### Pre-build steps

Before starting the build process, review the [inventory](./installer/inventory) file, and uncomment and provide values for the following variables found in the `[all:vars]` section uncommenting when necessary. Make sure the openshift and standalone docker sections are commented out:

*kubernetes_context*

> Prior to running the installer, make sure you've configured the context for the cluster you'll be installing to. This is how the installer knows which cluster to connect to and what authentication to use
*awx_kubernetes_namespace*

> Name of the Kubernetes namespace where the AWX resources will be installed. This will be created if it doesn't exist
*docker_registry_*

> These settings should be used if building your own base images. You'll need access to an external registry and are responsible for making sure your kube cluster can talk to it and use it. If these are undefined and the dockerhub_ configuration settings are uncommented then the images will be pulled from dockerhub instead
### Start the build

After making changes to the `inventory` file use `ansible-playbook` to begin the install

```bash
$ ansible-playbook -i inventory install.yml
```

### Post build

After the playbook run completes, check the status of the deployment by running `kubectl get pods --namespace awx` (replace awx with the namespace you used):

```bash
# View the running pods, it may take a few minutes for everything to be marked in the Running state
$ kubectl get pods --namespace awx
NAME READY STATUS RESTARTS AGE
awx-2558692395-2r8ss 4/4 Running 0 29s
awx-postgresql-355348841-kltkn 1/1 Running 0 1m
```

### Accessing AWX

The AWX web interface is running in the AWX pod behind the `awx-web-svc` service:

```bash
# View available services
$ kubectl get svc --namespace awx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
awx-postgresql ClusterIP 10.7.250.208 <none> 5432/TCP 2m
awx-web-svc NodePort 10.7.241.35 <none> 80:30177/TCP 1m
```

The deployment process creates an `Ingress` named `awx-web-svc` also. Some kubernetes cloud providers will automatically handle routing configuration when an Ingress is created others may require that you more explicitly configure it. You can see what kubernetes knows about things with:

```bash
kubectl get ing --namespace awx
NAME HOSTS ADDRESS PORTS AGE
awx-web-svc * 35.227.x.y 80 3m
```

If your provider is able to allocate an IP Address from the Ingress controller then you can navigate to the address and access the AWX interface. For some providers it can take a few minutes to allocate and make this accessible. For other providers it may require you to manually intervene.

### SSL Termination

Unlike Openshift's `Route` the Kubernetes `Ingress` doesn't yet handle SSL termination. As such the default configuration will only expose AWX through HTTP on port 80. You are responsible for configuring SSL support until support is added (either to Kubernetes or AWX itself).

## Docker

### Prerequisites
Expand Down
3 changes: 2 additions & 1 deletion installer/install.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,5 @@
- { role: check_vars }
- { role: image_build, when: "dockerhub_base is not defined" }
- { role: openshift, when: "openshift_host is defined" }
- { role: local_docker, when: "openshift_host is not defined" }
- { role: kubernetes, when: "kubernetes_context is defined" }
- { role: local_docker, when: "openshift_host is not defined and kubernetes_context is not defined" }
4 changes: 4 additions & 0 deletions installer/inventory
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,10 @@ awx_secret_key=awxsecret
# openshift_user=developer
# awx_node_port=30083

# Kubernetes Install
# kubernetes_context=test-cluster
# awx_kubernetes_namespace=awx

# Standalone Docker Install
postgres_data_dir=/tmp/pgdocker
host_port=80
Expand Down
115 changes: 115 additions & 0 deletions installer/kubernetes/tasks/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
---
- name: Set the Kubernetes Context
shell: "kubectl config set-context {{ kubernetes_context }}"

- name: Get Namespace Detail
shell: "kubectl get namespace {{ awx_kubernetes_namespace }}"
register: namespace_details
ignore_errors: yes

- name: Get Postgres Service Detail
shell: "kubectl describe svc awx-postgresql -n {{ awx_kubernetes_namespace }}"
register: postgres_svc_details
ignore_errors: yes
when: "pg_hostname is not defined or pg_hostname == ''"

- name: Create AWX Kubernetes Project
shell: "kubectl create namespace {{ awx_kubernetes_namespace }}"
when: namespace_details.rc != 0

# TODO: This is duplicated in the openshift role, probably needs to be moved to the image_build role
- name: Manage AWX Container Images
block:
- name: Authenticate with Docker registry
docker_login:
registry: "{{ docker_registry }}"
username: "{{ docker_registry_username }}"
password: "{{ docker_registry_password }}"
reauthorize: yes
when: docker_registry is defined and docker_registry_password is defined
delegate_to: localhost

- name: Wait for Openshift
pause:
seconds: 30

- name: Tag and push web image to registry
docker_image:
name: "{{ awx_web_image }}"
repository: "{{ docker_registry }}/{{ docker_registry_repository }}/{{ awx_web_image }}"
tag: "{{ awx_version }}"
push: yes
when: docker_registry is defined
delegate_to: localhost

- name: Wait for the registry to settle
pause:
seconds: 10

- name: Tag and push task image to registry
docker_image:
name: "{{ awx_task_image }}"
repository: "{{ docker_registry }}/{{ docker_registry_repository }}/{{ awx_task_image }}"
tag: "{{ awx_version }}"
push: yes
when: docker_registry is defined
delegate_to: localhost

- name: Set full web image path
set_fact:
awx_web_kubernetes_image: "{{ awx_web_image }}:{{ awx_version }}"
when: awx_web_kubernetes_image is not defined

- name: Set full task image path
set_fact:
awx_task_kubernetes_image: "{{ awx_task_image }}:{{ awx_version }}"
when: awx_task_kubernetes_image is not defined
when: dockerhub_base is not defined

- name: Set DockerHub Image Paths
set_fact:
awx_web_kubernetes_image: "{{ dockerhub_base }}/awx_web:{{ dockerhub_version }}"
awx_task_kubernetes_image: "{{ dockerhub_base }}/awx_task:{{ dockerhub_version }}"
when: dockerhub_base is defined

- name: Deploy and Activate Postgres
shell: "helm install --name awx --namespace {{ awx_kubernetes_namespace }} --set postgresUser={{ pg_username }},postgresPassword={{ pg_password }},postgresDatabase={{ pg_database }},persistence.size={{ pg_volume_capacity|default('5')}}Gi stable/postgresql"
when: (pg_hostname is not defined or pg_hostname == '') and (postgres_svc_details is defined and postgres_svc_details.rc != 0)
register: kubernetes_pg_activate

- name: Set postgresql hostname to helm package service
set_fact:
pg_hostname: awx-postgresql
when: pg_hostname is not defined or pg_hostname == ''

- name: Wait for Postgres to activate
pause:
seconds: 60
when: kubernetes_pg_activate|changed

- name: Set kubernetes base path
set_fact:
kubernetes_base_path: "{{ awx_local_base_config_path|default('/tmp') }}/awx-config"

- name: Ensure directory exists
file:
path: "{{ kubernetes_base_path }}"
state: directory

- name: Template Kubernetes AWX Config
template:
src: configmap.yml.j2
dest: "{{ kubernetes_base_path }}/configmap.yml"
mode: '0600'

- name: Template Kubernetes AWX Deployment
template:
src: deployment.yml.j2
dest: "{{ kubernetes_base_path }}/deployment.yml"
mode: '0600'

- name: Apply Configmap
shell: "kubectl apply -f {{ kubernetes_base_path }}/configmap.yml"

- name: Apply Deployment
shell: "kubectl apply -f {{ kubernetes_base_path }}/deployment.yml"
95 changes: 95 additions & 0 deletions installer/kubernetes/templates/configmap.yml.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: awx-config
namespace: {{ awx_kubernetes_namespace }}
data:
secret_key: {{ awx_secret_key }}
awx_settings: |
import os
import socket
ADMINS = ()

# Container environments don't like chroots
AWX_PROOT_ENABLED = False

#Autoprovisioning should replace this
CLUSTER_HOST_ID = socket.gethostname()
SYSTEM_UUID = '00000000-0000-0000-0000-000000000000'

CELERY_TASK_QUEUES += (Queue(CLUSTER_HOST_ID, Exchange(CLUSTER_HOST_ID), routing_key=CLUSTER_HOST_ID),)
CELERY_TASK_ROUTES['awx.main.tasks.cluster_node_heartbeat'] = {'queue': CLUSTER_HOST_ID, 'routing_key': CLUSTER_HOST_ID}
CELERY_TASK_ROUTES['awx.main.tasks.purge_old_stdout_files'] = {'queue': CLUSTER_HOST_ID, 'routing_key': CLUSTER_HOST_ID}
STATIC_ROOT = '/var/lib/awx/public/static'
PROJECTS_ROOT = '/var/lib/awx/projects'
JOBOUTPUT_ROOT = '/var/lib/awx/job_status'
SECRET_KEY = file('/etc/tower/SECRET_KEY', 'rb').read().strip()
ALLOWED_HOSTS = ['*']
INTERNAL_API_URL = 'http://127.0.0.1:8052'
AWX_TASK_ENV['HOME'] = '/var/lib/awx'
SERVER_EMAIL = 'root@localhost'
DEFAULT_FROM_EMAIL = 'webmaster@localhost'
EMAIL_SUBJECT_PREFIX = '[AWX] '
EMAIL_HOST = 'localhost'
EMAIL_PORT = 25
EMAIL_HOST_USER = ''
EMAIL_HOST_PASSWORD = ''
EMAIL_USE_TLS = False

LOGGING['handlers']['console'] = {
'()': 'logging.StreamHandler',
'level': 'DEBUG',
'formatter': 'simple',
}

LOGGING['loggers']['django.request']['handlers'] = ['console']
LOGGING['loggers']['rest_framework.request']['handlers'] = ['console']
LOGGING['loggers']['awx']['handlers'] = ['console']
LOGGING['loggers']['awx.main.commands.run_callback_receiver']['handlers'] = ['console']
LOGGING['loggers']['awx.main.commands.inventory_import']['handlers'] = ['console']
LOGGING['loggers']['awx.main.tasks']['handlers'] = ['console']
LOGGING['loggers']['awx.main.scheduler']['handlers'] = ['console']
LOGGING['loggers']['django_auth_ldap']['handlers'] = ['console']
LOGGING['loggers']['social']['handlers'] = ['console']
LOGGING['loggers']['system_tracking_migrations']['handlers'] = ['console']
LOGGING['loggers']['rbac_migrations']['handlers'] = ['console']
LOGGING['loggers']['awx.isolated.manager.playbooks']['handlers'] = ['console']
LOGGING['handlers']['callback_receiver'] = {'class': 'logging.NullHandler'}
LOGGING['handlers']['fact_receiver'] = {'class': 'logging.NullHandler'}
LOGGING['handlers']['task_system'] = {'class': 'logging.NullHandler'}
LOGGING['handlers']['tower_warnings'] = {'class': 'logging.NullHandler'}
LOGGING['handlers']['rbac_migrations'] = {'class': 'logging.NullHandler'}
LOGGING['handlers']['system_tracking_migrations'] = {'class': 'logging.NullHandler'}
LOGGING['handlers']['management_playbooks'] = {'class': 'logging.NullHandler'}

DATABASES = {
'default': {
'ATOMIC_REQUESTS': True,
'ENGINE': 'django.db.backends.postgresql',
'NAME': "{{ pg_database }}",
'USER': "{{ pg_username }}",
'PASSWORD': "{{ pg_password }}",
'HOST': "{{ pg_hostname|default('postgresql') }}",
'PORT': "{{ pg_port }}",
}
}
CELERY_BROKER_URL = 'amqp://{}:{}@{}:{}/{}'.format(
"awx",
"abcdefg",
"localhost",
"5672",
"awx")
CHANNEL_LAYERS = {
'default': {'BACKEND': 'asgi_amqp.AMQPChannelLayer',
'ROUTING': 'awx.main.routing.channel_routing',
'CONFIG': {'url': CELERY_BROKER_URL}}
}
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': '{}:{}'.format("localhost", "11211")
},
'ephemeral': {
'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
},
}
Loading

0 comments on commit ae06cff

Please sign in to comment.