Skip to content

Commit

Permalink
Performance tuning, deprecation updates, and more
Browse files Browse the repository at this point in the history
- Add performance tuning configuration (thanks @t0k4rt)
  - Set raft multiplier to 1
- Conditionally install Python dependency baed on virtualenv or --user
  Addresses #129 (comment)
- Update includes to import_tasks and include_tasks
- Remove invalid consul_version key from configuration
- Update Vagrantfile
  - Set client address to 0.0.0.0 so Vagrant based deploy checks now pass
- Update documentation
  • Loading branch information
brianshumate committed Jan 11, 2018
1 parent 0641753 commit 3f01ed0
Show file tree
Hide file tree
Showing 9 changed files with 68 additions and 80 deletions.
8 changes: 6 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -768,10 +768,14 @@ Actually add new template files :facepalm:
- Consul v1.0.2
- Update documentation

## v1.30.1 (UNRELEASED)
## v1.30.1

- Add performance tuning configuration (thanks @t0k4rt)
- Set raft multiplier to 1
- Update documentation
- Conditionally install Python dependency baed on virtualenv or --user
Addresses https://github.com/brianshumate/ansible-consul/issues/129#issuecomment-356095611
- Update includes to import_tasks and include_tasks
- Remove invalid consul_version key from configuration
- Update Vagrantfile
- Set client address to 0.0.0.0 so Vagrant based deploy checks now pass
- Update documentation
38 changes: 22 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ definitely known to work with the following specific software versions:
* Consul: 1.0.2
* Ansible: 2.4.2.0
* CentOS: 7
* Debian: 8
* Debian: 9
* FreeBSD: 11
* RHEL: 7
* Ubuntu: 16.04
Expand Down Expand Up @@ -667,6 +667,27 @@ packages with different package names.
- List of OS packages to install
- Default value: list

### `consul_performance`

- List of Consul performance tuning items
- Default value: list

#### `raft_multiplier`

- [Raft multiplier](https://www.consul.io/docs/agent/options.html#raft_multiplier) scales key Raft timing parameters
- Default value: 1

#### `leave_drain_time`

- [Node leave drain time](https://www.consul.io/docs/agent/options.html#leave_drain_time) is the dwell time for a server to honor requests while gracefully leaving

- Default value: 5s

#### `rpc_hold_timeout`

- [RPC hold timeout](https://www.consul.io/docs/agent/options.html#rpc_hold_timeout) is the duration that a client or server will retry internal RPC requests during leader elections
- Default value: 7s

## Dependencies

Ansible requires GNU tar and this role performs some local use of the
Expand Down Expand Up @@ -813,21 +834,6 @@ consul3.node.consul. 0 IN A 10.1.42.230
- Custom list of addresses to listen on.
- Default value: *[]*

### `consul_performance`

This is a nested object of Consul performance related options.

#### `consul_performance.raft_multiplier`

This is the [raft multiplier](https://www.consul.io/docs/agent/options.html#raft_multiplier). See also [Server Performance](https://www.consul.io/docs/guides/performance.html).

#### `consul_performance.leave_drain_time`

This is the [leave drain time](https://www.consul.io/docs/agent/options.html#leave_drain_time).

#### `consul_performance.rpc_hold_timeout`

This is the [RPC hold timeout](https://www.consul.io/docs/agent/options.html#rpc_hold_timeout).

### iptables DNS Forwarding Support

Expand Down
2 changes: 1 addition & 1 deletion defaults/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ consul_dnsmasq_local_service: false
consul_dnsmasq_listen_addresses: []
consul_iptables_enable: "{{ lookup('env','CONSUL_IPTABLES_ENABLE') | default(false, true) }}"

# Consul performace
# Performance
consul_performance:
raft_multiplier: 1
leave_drain_time: 5s
Expand Down
39 changes: 10 additions & 29 deletions examples/README_VAGRANT.md
Original file line number Diff line number Diff line change
@@ -1,55 +1,38 @@
# Consul with Ansible

This project provides documentation and a collection of scripts to help you
automate the deployment of Consul using
[Ansible](https://www.ansible.com/). These are the instructions for
deploying a development cluster on Vagrant and VirtualBox.
This project provides documentation and a collection of scripts to help you automate the deployment of Consul using [Ansible](https://www.ansible.com/). These are the instructions for deploying a development cluster on Vagrant and VirtualBox.

The documentation and scripts are merely a starting point designed to both
help familiarize you with the processes and quickly bootstrap an environment
for development. You may wish to expand on them and customize
them with additional features specific to your needs later.
The documentation and scripts are merely a starting point designed to both help familiarize you with the processes and quickly bootstrap an environment for development. You may wish to expand on them and customize them with additional features specific to your needs later.

If you are looking for the main role documentation, it is in the [README.md](https://github.com/brianshumate/ansible-consul/blob/master/README.md).

## Vagrant Development Cluster

In some situations deploying a small cluster on your local development
machine can be handy. This document describes such a scenario using the
following technologies:
In some situations deploying a small cluster on your local development machine can be handy. This document describes such a scenario using the following technologies:

* [Consul](https://consul.io)
* [VirtualBox](https://www.virtualbox.org/)
* [Vagrant](http://www.vagrantup.com/) with Ansible provisioner and
supporting plugin
* [Ansible](https://www.ansible.com/)

Each of the virtual machines for this guide are configured with
1GB RAM, 2 CPU cores, and 2 network interfaces. The first interface uses
NAT and has connection via the host to the outside world. The second
interface is a private network and is used for Consul intra-cluster
communication in addition to access from the host machine.
Each of the virtual machines for this guide are configured with 1GB RAM, 2 CPU cores, and 2 network interfaces. The first interface uses NAT and has connection via the host to the outside world. The second interface is a private network and is used for Consul intra-cluster communication in addition to access from the host machine.

The Vagrant configuration file (`Vagrantfile`) is responsible for
configuring the virtual machines and a baseline OS installation.
The Vagrant configuration file (`Vagrantfile`) is responsible for configuring the virtual machines and a baseline OS installation.

The Ansible playbooks then further refine OS configuration, perform Consul
software download, installation, configuration, and the joining of server nodes into a ready to use cluster.
The Ansible playbooks then further refine OS configuration, perform Consul software download, installation, configuration, and the joining of server nodes into a ready to use cluster.

## Designed for Ansible Galaxy

This role is designed to be installed via the `ansible-galaxy` command
instead of being directly run from the git repository.
This role is designed to be installed via the `ansible-galaxy` command instead of being directly run from the git repository.

You should install it like this:

```
ansible-galaxy install brianshumate.consul
```

You'll want to make sure you have write access to `/etc/ansible/roles/` since
that is where the role will be installed by default, or define your own
Ansible role path by creating a `$HOME/.ansible.cfg` or even `./anisible.cfg`
You'll want to make sure you have write access to `/etc/ansible/roles/` since that is where the role will be installed by default, or define your own Ansible role path by creating a `$HOME/.ansible.cfg` or even `./anisible.cfg`
file with these contents:

```
Expand All @@ -61,8 +44,7 @@ Change `PATH_TO_ROLES` to a directory that you have write access to.

## Quick Start

Begin from the top level directory of this project and use the following
steps to get up and running:
Begin from the top level directory of this project and use the following steps to get up and running:

1. Install [VirtualBox](https://www.virtualbox.org/wiki/Downloads), [Vagrant](http://downloads.vagrantup.com/), [vagrant-hosts](https://github.com/adrienthebo/vagrant-hosts), and [Ansible](http://docs.ansible.com/ansible/intro_installation.html).
2. Edit `/etc/hosts` or use the included `bin/preinstall` script to add
Expand Down Expand Up @@ -113,8 +95,7 @@ BOX_NAME=freebsd/FreeBSD-11.0-STABLE CONSUL_IFACE=em1 vagrant up
* VirtualBox version 5.1.30
* Vagrant version 2.0.1
* Vagrant Hosts plugin version 2.8.0
2. This project uses Debian 8 (Jessie) by default, but you can choose another
OS distribution with the *BOX_NAME* environment variable
2. This project uses Debian 9 (Stretch) by default, but you can choose another OS distribution with the *BOX_NAME* environment variable
3. The `bin/preinstall` shell script performs the following actions for you:
* Adds each node's host information to the host machine's `/etc/hosts`
* Optionally installs the Vagrant hosts plugin
Expand Down
6 changes: 4 additions & 2 deletions examples/Vagrantfile
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ SCRIPT

ANSIBLE_PLAYBOOK = ENV['ANSIBLE_PLAYBOOK'] || "site.yml"
BOX_MEM = ENV['BOX_MEM'] || "1024"
BOX_NAME = ENV['BOX_NAME'] || "debian/jessie64"
BOX_NAME = ENV['BOX_NAME'] || "debian/stretch64"
CLUSTER_HOSTS = ENV['CLUSTER_HOSTS'] || "vagrant_hosts"
CONSUL_ACL_ENABLE = ENV['CONSUL_ACL_ENABLE'] || "false"
CONSUL_ATLAS_ENABLE = ENV['CONSUL_ATLAS_ENABLE'] || "false"
Expand Down Expand Up @@ -139,10 +139,12 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
# extra Ansible variables can be defined here as well. Wheeee!
#
ansible.extra_vars = {
consul_log_level: "DEBUG"
consul_log_level: "DEBUG",
consul_client_address: "10.1.42.230"
}
ansible.playbook = ANSIBLE_PLAYBOOK
ansible.limit = "all"
compatibility_mode = "2.0"
end
end
end
4 changes: 2 additions & 2 deletions handlers/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
# File: main.yml - Handlers for Consul

- name: restart consul
include: restart_consul.yml
import_tasks: restart_consul.yml

- name: start consul
include: start_consul.yml
import_tasks: start_consul.yml

- name: restart dnsmasq
service:
Expand Down
40 changes: 19 additions & 21 deletions tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
when: is_virtualenv is defined

- name: Include checks/asserts
include: asserts.yml
import_tasks: asserts.yml

- name: Include OS-specific variables
include_vars: "{{ ansible_os_family }}.yml"
Expand Down Expand Up @@ -55,10 +55,10 @@
tags: always

- name: Include user and group settings
include: user_group.yml
import_tasks: user_group.yml

- name: Include directory settings
include: dirs.yml
import_tasks: dirs.yml

- name: Check for existing Consul binary
stat:
Expand All @@ -70,19 +70,19 @@
consul_install_binary: "{{ consul_install_upgrade or not consul_binary_installed.stat.exists }}"

- name: Install OS packages and consul - locally
include: install.yml
include_tasks: install.yml
when:
- consul_install_binary | bool
- not consul_install_remotely | bool

- name: Install OS packages and consul - remotely
include: install_remote.yml
include_tasks: install_remote.yml
when:
- consul_install_binary | bool
- consul_install_remotely | bool

# XXX: Individual gossip tasks are deprecated and need to be removed
# - include: ../tasks/encrypt_gossip.yml
# - include_tasks: ../tasks/encrypt_gossip.yml
- block:
- block:
- name: Check for gossip encryption key on previously boostrapped server
Expand Down Expand Up @@ -159,14 +159,14 @@
- consul_encrypt_enable

- name: Create Consul configuration
include: config.yml
import_tasks: config.yml

- name: Create ACL configuration
include: acl.yml
include_tasks: acl.yml
when: consul_acl_enable | bool

- name: Create TLS configuration
include: tls.yml
include_tasks: tls.yml
when: consul_tls_enable | bool

- block:
Expand Down Expand Up @@ -240,7 +240,6 @@
- smfmanifest | changed
- ansible_os_family == "Solaris"


- name: Start Consul
service:
name: consul
Expand All @@ -250,7 +249,7 @@
- name: Check Consul HTTP API (via TCP socket)
wait_for:
delay: 15
port: "{{consul_ports.http|int}}"
port: "{{ consul_ports.http|int }}"
host: "{{ consul_addresses.http }}"
when: (consul_ports.http|int > -1) and (consul_addresses.http|ipaddr)

Expand All @@ -265,10 +264,10 @@
dest: "{{ consul_bootstrap_state }}"
state: touch

- include: ../tasks/dnsmasq.yml
- include_tasks: ../tasks/dnsmasq.yml
when: consul_dnsmasq_enable | bool

- include: ../tasks/iptables.yml
- include_tasks: ../tasks/iptables.yml
when: consul_iptables_enable | bool

when: not bootstrap_state.stat.exists
Expand Down Expand Up @@ -310,15 +309,15 @@
tags: always

- name: (Windows) Include directory settings
include: dirs.yml
import_tasks: dirs.yml

- name: (Windows) Check for existing Consul binary
win_stat:
path: "{{ consul_binary }}"
register: consul_binary_installed

- name: (Windows) Install OS packages and consul
include: install_windows.yml
include_tasks: install_windows.yml
when:
- not consul_binary_installed.stat.exists | bool

Expand Down Expand Up @@ -397,7 +396,7 @@
- consul_encrypt_enable

- name: (Windows) Create Consul configuration
include: config_windows.yml
import_tasks: config_windows.yml

- name: (Windows) Ensure neither ACL nor TLS are requested
fail:
Expand All @@ -406,11 +405,11 @@
- (consul_acl_enable | bool) or (consul_tls_enable | bool)

- name: (Windows) Create ACL configuration
include: acl.yml
include_tasks: acl.yml
when: consul_acl_enable | bool

- name: (Windows) Create TLS configuration
include: tls.yml
include_tasks: tls.yml
when: consul_tls_enable | bool

- block:
Expand All @@ -436,17 +435,16 @@
host: "{{inventory_hostname_short}}"
port: 8500


- name: (Windows) Create bootstrapped state file
win_file:
dest: "{{consul_bootstrap_state}}"
state: touch
when: ansible_os_family == "Windows"

- include: ../tasks/dnsmasq.yml
- include_tasks: ../tasks/dnsmasq.yml
when: consul_dnsmasq_enable | bool

- include: ../tasks/iptables.yml
- include_tasks: ../tasks/iptables.yml
when: consul_iptables_enable | bool

when: not bootstrap_state.stat.exists
Expand Down
9 changes: 3 additions & 6 deletions templates/config.json.j2
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,9 @@
"node_name": "{{ consul_node_name }}",
"datacenter": "{{ consul_datacenter }}",
"domain": "{{ consul_domain }}",
"consul_version": "{{consul_version}}",
{% if consul_version | version_compare('0.7.3', '>=') and
consul_node_meta | length > 0 %}
{% if consul_version | version_compare('0.7.3', '>=') and consul_node_meta | length > 0 %}
"node_meta": {{ consul_node_meta | default({})| to_json }},
{% endif %}

{# Performance Settings #}
"performance": {{ consul_performance | to_json }},

Expand All @@ -28,7 +25,7 @@
"dns": "{{ consul_addresses.dns }}",
"http": "{{ consul_addresses.http }}",
"https": "{{ consul_addresses.https }}"

},
{## Ports Used ##}
"ports": {
Expand Down Expand Up @@ -90,7 +87,7 @@
{## LAN Join ##}
"retry_interval": "{{ consul_retry_interval }}",
"retry_max": {{ consul_retry_max | int }},
"retry_join":
"retry_join":
{% for server in _consul_lan_servers %}
{% set consul_join = consul_join + [ hostvars[server]['consul_advertise_address'] | default(hostvars[server]['consul_bind_address']) ] %}{% if loop.last %}
{{ consul_join | map('ipwrap') | list | to_json }},
Expand Down
2 changes: 1 addition & 1 deletion version.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
v1.30.0
v1.30.1

0 comments on commit 3f01ed0

Please sign in to comment.