diff --git a/website/content/commands/agent.mdx b/website/content/commands/agent.mdx index ccfc991d8f9..ff30b22a067 100644 --- a/website/content/commands/agent.mdx +++ b/website/content/commands/agent.mdx @@ -35,7 +35,7 @@ information. limit of 4k for maximum size of checks, this is a positive value. By limiting this size, it allows to put less pressure on Consul servers when many checks are having a very large output in their checks. In order to completely disable check output - capture, it is possible to use [`discard_check_output`](/consul/docs/reference/agent#discard_check_output). + capture, it is possible to use [`discard_check_output`](/consul/docs/reference/agent/configuration-file/general#discard_check_output). - `-client` ((#\_client)) - The address to which Consul will bind client interfaces, including the HTTP and DNS servers. By default, this is "127.0.0.1", @@ -154,7 +154,7 @@ information. - `-raft-protocol` ((#\_raft_protocol)) - This controls the internal version of the Raft consensus protocol used for server communications. This must be set - to 3 in order to gain access to Autopilot features, with the exception of [`cleanup_dead_servers`](/consul/docs/reference/agent#cleanup_dead_servers). Defaults to 3 in Consul 1.0.0 and later (defaulted to 2 previously). See [Raft Protocol Version Compatibility](/consul/docs/upgrade/version-specific#raft-protocol-version-compatibility) for more details. + to 3 in order to gain access to Autopilot features, with the exception of [`cleanup_dead_servers`](/consul/docs/reference/agent/configuration-file/general#cleanup_dead_servers). Defaults to 3 in Consul 1.0.0 and later (defaulted to 2 previously). See [Raft Protocol Version Compatibility](/consul/docs/upgrade/version-specific#raft-protocol-version-compatibility) for more details. - `-segment` ((#\_segment)) - This flag is used to set the name of the network segment the agent belongs to. An agent can only join and @@ -186,13 +186,13 @@ information. - `-advertise-wan` ((#\_advertise-wan)) - The advertise WAN address is used to change the address that we advertise to server nodes joining through the WAN. - This can also be set on client agents when used in combination with the [`translate_wan_addrs`](/consul/docs/reference/agent#translate_wan_addrs) configuration option. By default, the [`-advertise`](#_advertise) address + This can also be set on client agents when used in combination with the [`translate_wan_addrs`](/consul/docs/reference/agent/configuration-file/general#translate_wan_addrs) configuration option. By default, the [`-advertise`](#_advertise) address is advertised. However, in some cases all members of all datacenters cannot be on the same physical or virtual network, especially on hybrid setups mixing cloud and private datacenters. This flag enables server nodes gossiping through the public network for the WAN while using private VLANs for gossiping to each other and their client agents, and it allows client agents to be reached at this address when being - accessed from a remote datacenter if the remote datacenter is configured with [`translate_wan_addrs`](/consul/docs/reference/agent#translate_wan_addrs). In Consul 1.1.0 and later this can be dynamically defined with a [go-sockaddr] + accessed from a remote datacenter if the remote datacenter is configured with [`translate_wan_addrs`](/consul/docs/reference/agent/configuration-file/general#translate_wan_addrs). In Consul 1.1.0 and later this can be dynamically defined with a [go-sockaddr] template that is resolved at runtime. ## Address Bind Options @@ -306,7 +306,7 @@ information. - `-recursor` ((#\_recursor)) - Specifies the address of an upstream DNS server. This option may be provided multiple times, and is functionally equivalent - to the [`recursors` configuration option](/consul/docs/reference/agent#recursors). + to the [`recursors` configuration option](/consul/docs/reference/agent/configuration-file/general#recursors). - `-join` ((#\_join)) - **Deprecated in Consul 1.15. This flag will be removed in a future version of Consul. Use the `-retry-join` flag instead.** This is an alias of [`-retry-join`](#_retry_join). @@ -323,7 +323,7 @@ information. This can be dynamically defined with a [go-sockaddr] template that is resolved at runtime. - If Consul is running on a non-default Serf LAN port, you must specify the port number in the address when using the `-retry-join` flag. Alternatively, you can specify the custom port number as the default in the agent's [`ports.serf_lan`](/consul/docs/reference/agent#serf_lan_port) configuration or with the [`-serf-lan-port`](#_serf_lan_port) command line flag when starting the agent. + If Consul is running on a non-default Serf LAN port, you must specify the port number in the address when using the `-retry-join` flag. Alternatively, you can specify the custom port number as the default in the agent's [`ports.serf_lan`](/consul/docs/reference/agent/configuration-file/general#serf_lan_port) configuration or with the [`-serf-lan-port`](#_serf_lan_port) command line flag when starting the agent. If your network contains network segments, refer to the [network segments documentation](/consul/docs/multi-tenant/network-segment/vm) for additional information. diff --git a/website/content/docs/automate/kv/index.mdx b/website/content/docs/automate/kv/index.mdx index 7e83b813a60..720cfd54cb6 100644 --- a/website/content/docs/automate/kv/index.mdx +++ b/website/content/docs/automate/kv/index.mdx @@ -25,11 +25,11 @@ The datastore itself is located on the Consul servers in the [data directory](/c ## Using Consul KV -Objects are opaque to Consul, meaning there are no restrictions on the type of object stored in a key/value entry. The main restriction on an object is size - the maximum is 512 KB. Due to the maximum object size and main use cases, you should not need extra storage; the general [sizing recommendations](/consul/docs/reference/agent#kv_max_value_size) are usually sufficient. +Objects are opaque to Consul, meaning there are no restrictions on the type of object stored in a key/value entry. The main restriction on an object is size - the maximum is 512 KB. Due to the maximum object size and main use cases, you should not need extra storage; the general [sizing recommendations](/consul/docs/reference/agent/configuration-file/general#kv_max_value_size) are usually sufficient. Keys, like objects are not restricted by type and can include any character. However, we recommend using URL-safe chars - `[a-zA-Z0-9-._~]` with the exception of `/`, which can be used to help organize data. Note, `/` will be treated like any other character and is not fixed to the file system. Meaning, including `/` in a key does not fix it to a directory structure. This model is similar to Amazon S3 buckets. However, `/` is still useful for organizing data and when recursively searching within the data store. We also recommend that you avoid the use of `*`, `?`, `'`, and `%` because they can cause issues when using the API and in shell scripts. -## Using Sentinel to apply policies for Consul KV +## Using Sentinel to apply policies for Consul KV @@ -59,4 +59,5 @@ Review the following tutorials to learn how to use Consul sessions for [applicat ### Vault -If you plan to use Consul KV as a backend for Vault, please review [this tutorial](/vault/tutorials/day-one-consul/ha-with-consul?utm_source=docs). \ No newline at end of file +If you plan to use Consul KV as a backend for Vault, please review [this +tutorial](/vault/tutorials/day-one-consul/ha-with-consul?utm_source=docs). diff --git a/website/content/docs/connect/proxy/sidecar.mdx b/website/content/docs/connect/proxy/sidecar.mdx index 0d483627721..543f579cf94 100644 --- a/website/content/docs/connect/proxy/sidecar.mdx +++ b/website/content/docs/connect/proxy/sidecar.mdx @@ -11,15 +11,15 @@ This topic describes how to create, register, and start sidecar proxy services i ## Overview -Sidecar proxies run on the same node as the single service instance that they handle traffic for. -They may be on the same VM or running as a separate container in the same network namespace. +Sidecar proxies run on the same node as the single service instance that they handle traffic for. +They may be on the same VM or running as a separate container in the same network namespace. You can attach a sidecar proxy to a service you want to deploy to your mesh: -1. It is not required, but you can create a proxy defaults configuration entry that contains global passthrough settings for all Envoy proxies. +1. It is not required, but you can create a proxy defaults configuration entry that contains global passthrough settings for all Envoy proxies. 1. Create the service definition and include the `connect` block. The `connect` block contains the sidecar proxy configurations that allow the service to interact with other services in the mesh. 1. Register the service using either the API or CLI. -1. Start the sidecar proxy service. +1. Start the sidecar proxy service. ## Requirements @@ -32,21 +32,21 @@ If you want to define global passthrough settings for all Envoy proxies, create 1. Create a proxy defaults configuration entry and specify the following parameters: - `Kind`: Must be set to `proxy-defaults` - `Name`: Must be set to `global` -1. Configure any additional settings you want to apply to all proxies. Refer to [Proxy defaults configuration entry reference](/consul/docs/reference/config-entry/proxy-defaults) for details about all settings available in the configuraiton entry. +1. Configure any additional settings you want to apply to all proxies. Refer to [Proxy defaults configuration entry reference](/consul/docs/reference/config-entry/proxy-defaults) for details about all settings available in the configuraiton entry. 1. Apply the configuration by either calling the [`/config` API endpoint](/consul/api-docs/config) or running the [`consul config write` CLI command](/consul/commands/config/write). The following example writes a proxy defaults configuration entry from a local HCL file using the CLI: ```shell-session $ consul config write proxy-defaults.hcl ``` -## Define service mesh proxy +## Define service mesh proxy -Create a service definition and configure the following fields: +Create a service definition and configure the following fields: -1. `name`: Specify a name for the service you want to attach a sidecar proxy to in the `name` field. This field is required for all services you want to register in Consul. +1. `name`: Specify a name for the service you want to attach a sidecar proxy to in the `name` field. This field is required for all services you want to register in Consul. 1. `port`: Specify a port number where other services registered with Consul can discover and connect to the service in the `port` field. This field is required for all services you want to register in Consul. 1. `connect`: Set the `connect` field to `{ sidecar_service: {} }`. The `{ sidecar_service: {} }` value is a macro that applies a set of default configurations that enable you to quickly implement a sidecar. Refer to [Sidecar service defaults](#sidecar-service-defaults) for additional information. -1. Configure any additional options for your service. Refer to [Services configuration reference](/consul/docs/reference/service) for details. +1. Configure any additional options for your service. Refer to [Services configuration reference](/consul/docs/reference/service) for details. In the following example, a service named `web` is configured with a sidecar proxy: @@ -60,7 +60,7 @@ service = { port = 8080 connect = { sidecar_service = {} } } -``` +``` @@ -89,7 +89,7 @@ When Consul processes the service definition, it generates the following configu ```hcl -services = [ +services = [ { name = "web" port = 8080 @@ -114,7 +114,7 @@ services = [ } ] -``` +``` @@ -156,12 +156,12 @@ services = [ - + -## Register the service +## Register the service Provide the service definition to the Consul agent to register your proxy service. You can use the same methods for registering proxy services as you do for registering application services: - + - Place the service definition in a Consul agent's configuration directory and start, restart, or reload the agent. Use this method when implementing changes to an existing proxy service. - Use the `consul services register` command to register the proxy service with a running Consul agent. - Call the `/agent/service/register` HTTP API endpoint to register the proxy service with a running Consul agent. @@ -176,7 +176,7 @@ $ consul services register proxy.hcl ## Start the proxy -Envoy requires a bootstrap configuration file before it can start. Use the [`consul connect envoy` command](/consul/commands/connect/envoy) to create the Envoy bootstrap configuration and start the proxy service. Specify the name of the service with the attached proxy with the `-sidecar-for` option. +Envoy requires a bootstrap configuration file before it can start. Use the [`consul connect envoy` command](/consul/commands/connect/envoy) to create the Envoy bootstrap configuration and start the proxy service. Specify the name of the service with the attached proxy with the `-sidecar-for` option. The following example command starts an Envoy sidecar proxy for the `web` service: @@ -184,11 +184,11 @@ The following example command starts an Envoy sidecar proxy for the `web` servic $ consul connect envoy -sidecar-for=web ``` -For details about operating an Envoy proxy in Consul, refer to [](/consul/docs/reference/proxy/envoy) +For details about operating an Envoy proxy in Consul, refer to [](/consul/docs/reference/proxy/envoy) ## Configuration reference -The `sidecar_service` block is a service definition that can contain most regular service definition fields. Refer to [Limitations](#limitations) for information about unsupported service definition fields for sidecar proxies. +The `sidecar_service` block is a service definition that can contain most regular service definition fields. Refer to [Limitations](#limitations) for information about unsupported service definition fields for sidecar proxies. Consul treats sidecar proxy service definitions as a root-level service definition. All fields are optional in nested definitions, which default to opinionated settings that are intended to reduce burden of setting up a sidecar proxy. @@ -207,8 +207,8 @@ proxy. - `tags` - Defaults to the tags of the parent service. - `meta` - Defaults to the service metadata of the parent service. - `port` - Defaults to being auto-assigned from a configurable - range specified by [`sidecar_min_port`](/consul/docs/reference/agent#sidecar_min_port) - and [`sidecar_max_port`](/consul/docs/reference/agent#sidecar_max_port). + range specified by [`sidecar_min_port`](/consul/docs/reference/agent/configuration-file/general#sidecar_min_port) + and [`sidecar_max_port`](/consul/docs/reference/agent/configuration-file/general#sidecar_max_port). - `kind` - Defaults to `connect-proxy`. This value cannot be overridden. - `check`, `checks` - By default we add a TCP check on the local address and port for the proxy, and a [service alias check](/consul/docs/register/health-check/vm#alias-checks) for the parent service. If either @@ -223,7 +223,7 @@ proxy. In the following example, the `sidecar_service` macro sets baselines configurations for the proxy, but the [proxy upstreams](/consul/docs/reference/proxy/connect-proxy#upstream-configuration-reference) and [built-in proxy -configuration](/consul/docs/reference/proxy/built-in) fields contain custom values: +configuration](/consul/docs/reference/proxy/built-in) fields contain custom values: ```json { @@ -280,4 +280,4 @@ service's ID, which enables the following behavior. - When reloading the configuration files, if a service definition changes its ID, then a new service instance and a new sidecar instance are registered. The old instance and proxy are removed because they are no longer found in - the configuration files. \ No newline at end of file + the configuration files. diff --git a/website/content/docs/deploy/server/cloud-auto-join.mdx b/website/content/docs/deploy/server/cloud-auto-join.mdx index e618160d628..8a2a0845ae1 100644 --- a/website/content/docs/deploy/server/cloud-auto-join.mdx +++ b/website/content/docs/deploy/server/cloud-auto-join.mdx @@ -63,7 +63,7 @@ A Consul client agent wishing to join the "alpha" segment would need to be confi -The following example configuration overrides the default Serf LAN port using the [`ports.serf_lan`](/consul/docs/reference/agent#serf_lan_port) configuration option. +The following example configuration overrides the default Serf LAN port using the [`ports.serf_lan`](/consul/docs/reference/agent/configuration-file/general#serf_lan_port) configuration option. @@ -207,7 +207,7 @@ that contains the virtual NICs for the Virtual Machines. When using Virtual Machine Scale Sets the only role action needed is `Microsoft.Compute/virtualMachineScaleSets/*/read`. - + If the Consul datacenter is hosted on Azure, Consul can use Managed Service Identities (MSI) to access Azure instead of an environment variable, shared client id and secret. MSI must be enabled on the VMs or Virtual Machine Scale Sets hosting Consul. It is the preferred configuration since MSI prevents your Azure credentials from being stored in Consul configuration. This feature is supported in Consul 1.7 and above. When using MSI, the `tag_key`, `tag_value` and `subscription_id` need to be supplied for Virtual machines. Be aware that the amount of time that Azure takes for the VMs to detect the MSI permissions can be between a minute to an hour. @@ -481,4 +481,4 @@ $ consul agent -retry-join "provider=linode region=us-east tag_name=consul-serve Variables can also be provided by environment variables: -- `LINODE_TOKEN` for `api_token` \ No newline at end of file +- `LINODE_TOKEN` for `api_token` diff --git a/website/content/docs/deploy/server/vm/bootstrap.mdx b/website/content/docs/deploy/server/vm/bootstrap.mdx index b1835031054..e87ed76fb32 100644 --- a/website/content/docs/deploy/server/vm/bootstrap.mdx +++ b/website/content/docs/deploy/server/vm/bootstrap.mdx @@ -40,7 +40,7 @@ You can also create an agent configuration file to use when deploying multiple C -```hcl +```hcl datacenter = "dc1" data_dir = "/tmp/consul" log_level = "INFO" @@ -64,7 +64,7 @@ Consul prints a warning message to the console when the number of servers in a c ## Join the servers -After you start the servers, you must join them in a cluster to initiate the Raft election. To join servers automatically, specify network addresses or [cloud auto join](/consul/docs/deploy/server/cloud-auto-join) tags for supported cloud environments using either the [-retry-join CLI flag](/consul/commands/agent#_retry_join) or the [`retry_join` configuration option](/consul/docs/reference/agent#retry_join). +After you start the servers, you must join them in a cluster to initiate the Raft election. To join servers automatically, specify network addresses or [cloud auto join](/consul/docs/deploy/server/cloud-auto-join) tags for supported cloud environments using either the [-retry-join CLI flag](/consul/commands/agent#_retry_join) or the [`retry_join` configuration option](/consul/docs/reference/agent/consul/docs/reference/agent/configuration-file/join#retry_join). The following examples demonstrate address options and their formatting for the `-retry-join` CLI flag. @@ -129,8 +129,8 @@ After you bootstrap a datacenter, you can make additional changes to the datacen We recommend removing `bootstrap_expect` from agent configurations and reloading the agents after the initial bootstrap process is complete. This action prevents server agents that fail from unintentionally bootstrapping again after they restart. Instead, they will rejoin a datacenter's cluster automatically. -You can also enable Consul's browser-based user interface, deploy client agents, and register services in the Consul catalog for service discovery and service mesh use cases. Refer to the following topics for more information: +You can also enable Consul's browser-based user interface, deploy client agents, and register services in the Consul catalog for service discovery and service mesh use cases. Refer to the following topics for more information: - [Consul UI visualization](/consul/docs/fundamentals/interface/ui) - [Configure client agents](/consul/docs/deploy/workload/client/vm) -- [Register service](/register/service/vm) \ No newline at end of file +- [Register service](/register/service/vm) diff --git a/website/content/docs/deploy/server/vm/requirements.mdx b/website/content/docs/deploy/server/vm/requirements.mdx index 818659110d1..35d6ddc9bd0 100644 --- a/website/content/docs/deploy/server/vm/requirements.mdx +++ b/website/content/docs/deploy/server/vm/requirements.mdx @@ -16,7 +16,7 @@ reads work from a fully in-memory data store that is optimized for concurrent ac ## Minimum Server Requirements ((#minimum)) -In Consul 0.7, the default server [performance parameters](/consul/docs/reference/agent#performance) +In Consul 0.7, the default server [performance parameters](/consul/docs/reference/agent/configuration-file/general#performance) were tuned to allow Consul to run reliably (but relatively slowly) on a server cluster of three [AWS t2.micro](https://aws.amazon.com/ec2/instance-types/) instances. These thresholds were determined empirically using a leader instance that was under sufficient read, write, @@ -41,7 +41,7 @@ The default performance configuration is equivalent to this: ## Production Server Requirements ((#production)) When running Consul 0.7 and later in production, it is recommended to configure the server -[performance parameters](/consul/docs/reference/agent#performance) back to Consul's original +[performance parameters](/consul/docs/reference/agent/configuration-file/general#performance) back to Consul's original high-performance settings. This will let Consul servers detect a failed leader and complete leader elections much more quickly than the default configuration which extends key Raft timeouts by a factor of 5, so it can be quite slow during these events. @@ -101,14 +101,14 @@ Here are some general recommendations: issues between the servers or insufficient CPU resources. Users in cloud environments often bump their servers up to the next instance class with improved networking and CPU until leader elections stabilize, and in Consul 0.7 or later the [performance - parameters](/consul/docs/reference/agent#performance) configuration now gives you tools + parameters](/consul/docs/reference/agent/configuration-file/general#performance) configuration now gives you tools to trade off performance instead of upsizing servers. You can use the [`consul.raft.leader.lastContact` telemetry](/consul/docs/reference/agent/telemetry#leadership-changes) to observe how the Raft timing is performing and guide the decision to de-tune Raft performance or add more powerful servers. - For DNS-heavy workloads, configuring all Consul agents in a cluster with the - [`allow_stale`](/consul/docs/reference/agent#allow_stale) configuration option will allow reads to + [`allow_stale`](/consul/docs/reference/agent/configuration-file/dns#allow_stale) configuration option will allow reads to scale across all Consul servers, not just the leader. Consul 0.7 and later enables stale reads for DNS by default. See [Stale Reads](/consul/tutorials/networking/dns-caching#stale-reads) in the [DNS Caching](/consul/tutorials/networking/dns-caching) guide for more details. It's also good to set @@ -158,7 +158,7 @@ Since writes must be synced to disk (persistent storage) on a quorum of servers For a **read-heavy** workload, configure all Consul server agents with the `allow_stale` DNS option, or query the API with the `stale` [consistency mode](/consul/api-docs/features/consistency). By default, all queries made to the server are RPC forwarded to and serviced by the leader. By enabling stale reads, any server will respond to any query, thereby reducing overhead on the leader. Typically, the stale response is `100ms` or less from consistent mode but it drastically improves performance and reduces latency under high load. -If the leader server is out of memory or the disk is full, the server eventually stops responding, loses its election and cannot move past its last commit time. However, by configuring `max_stale` and setting it to a large value, Consul will continue to respond to queries during such outage scenarios. ([max_stale documentation](/consul/docs/reference/agent#max_stale)). +If the leader server is out of memory or the disk is full, the server eventually stops responding, loses its election and cannot move past its last commit time. However, by configuring `max_stale` and setting it to a large value, Consul will continue to respond to queries during such outage scenarios. ([max_stale documentation](/consul/docs/reference/agent/configuration-file/dns#max_stale)). It should be noted that `stale` is not appropriate for coordination where strong consistency is important (i.e. locking or application leader election). For critical cases, the optional `consistent` API query mode is required for true linearizability; the trade off is that this turns a read into a full quorum write so requires more resources and takes longer. @@ -189,8 +189,8 @@ Smearing requests over 30s is sufficient to bring RPC load to a reasonable level in all but the very largest clusters, but the extra CPU load from cryptographic operations could impact the server's normal work. To limit that, Consul since 1.4.1 exposes two ways to limit the impact Certificate signing has on the leader -[`csr_max_per_second`](/consul/docs/reference/agent#ca_csr_max_per_second) and -[`csr_max_concurrent`](/consul/docs/reference/agent#ca_csr_max_concurrent). +[`csr_max_per_second`](/consul/docs/reference/agent/configuration-file/service-mesh#ca_csr_max_per_second) and +[`csr_max_concurrent`](/consul/docs/reference/agent/configuration-file/service-mesh#ca_csr_max_concurrent). By default we set a limit of 50 per second which is reasonable on modest hardware but may be too low and impact rotation times if more than 1500 service diff --git a/website/content/docs/discover/dns/configure.mdx b/website/content/docs/discover/dns/configure.mdx index 0048c1e11b6..d6c20ec8aa3 100644 --- a/website/content/docs/discover/dns/configure.mdx +++ b/website/content/docs/discover/dns/configure.mdx @@ -17,29 +17,29 @@ This topic describes the default behavior of the Consul DNS functionality and ho By default, the Consul DNS listens for queries at `127.0.0.1:8600` and uses the `consul` domain. Specify the following parameters in the agent configuration to determine DNS behavior when querying services: -- [`client_addr`](/consul/docs/reference/agent#client_addr) +- [`client_addr`](/consul/docs/reference/agent/configuration-file/general#client_addr) - [`ports.dns`](/consul/docs/reference/agent/configuration-file/general#dns_port): By default, Consul does not use port `53`, which is typically reserved for the default port for DNS resolvers, because it requires an escalated privilege to bind to. -- [`recursors`](/consul/docs/reference/agent#recursors) -- [`domain`](/consul/docs/reference/agent#domain) -- [`alt_domain`](/consul/docs/reference/agent#alt_domain) -- [`dns_config`](/consul/docs/reference/agent#dns_config) +- [`recursors`](/consul/docs/reference/agent/configuration-file/general#recursors) +- [`domain`](/consul/docs/reference/agent/configuration-file/dns#domain) +- [`alt_domain`](/consul/docs/reference/agent/configuration-file/general#alt_domain) +- [`dns_config`](/consul/docs/reference/agent/configuration-file/dns#dns_config) ### Configure WAN address translation By default, Consul DNS queries return a node's local address, even when being queried from a remote datacenter. You can configure the DNS to reach a node from outside its datacenter by specifying the address in the following configuration fields in the Consul agent: - [advertise-wan](/consul/commands/agent#_advertise-wan) -- [translate_wan_addrs](/consul/docs/reference/agent#translate_wan_addrs) +- [translate_wan_addrs](/consul/docs/reference/agent/configuration-file/general#translate_wan_addrs) ### Use a custom DNS resolver library -You can specify a list of addresses in the agent's [`recursors`](/consul/docs/reference/agent#recursors) field to provide upstream DNS servers that recursively resolve queries that are outside the service domain for Consul. +You can specify a list of addresses in the agent's [`recursors`](/consul/docs/reference/agent/configuration-file/general#recursors) field to provide upstream DNS servers that recursively resolve queries that are outside the service domain for Consul. Nodes that query records outside the `consul.` domain resolve to an upstream DNS. You can specify IP addresses or use `go-sockaddr` templates. Consul resolves IP addresses in the specified order and ignores duplicates. ### Enable non-Consul queries -You enable non-Consul queries to be resolved by setting Consul as the DNS server for a node and providing a [`recursors`](/consul/docs/reference/agent#recursors) configuration. +You enable non-Consul queries to be resolved by setting Consul as the DNS server for a node and providing a [`recursors`](/consul/docs/reference/agent/configuration-file/general##recursors) configuration. ### Forward queries to an agent @@ -47,9 +47,9 @@ You can forward all queries sent to the `consul.` domain from the existing DNS s ### Query an alternate domain -By default, Consul responds to DNS queries in the `consul` domain, but you can set a specific domain for responding to DNS queries by configuring the [`domain`](/consul/docs/reference/agent#domain) parameter. +By default, Consul responds to DNS queries in the `consul` domain, but you can set a specific domain for responding to DNS queries by configuring the [`domain`](/consul/docs/reference/agent/configuration-file/dns#domain) parameter. -You can also specify an additional domain in the [`alt_domain`](/consul/docs/reference/agent#alt_domain) agent configuration option, which configures Consul to respond to queries in a secondary domain. Configuring an alternate domain may be useful during a DNS migration or to distinguish between internal and external queries, for example. +You can also specify an additional domain in the [`alt_domain`](//consul/docs/reference/agent/configuration-file/general##alt_domain) agent configuration option, which configures Consul to respond to queries in a secondary domain. Configuring an alternate domain may be useful during a DNS migration or to distinguish between internal and external queries, for example. Consul's DNS response uses the same domain as the query. @@ -70,7 +70,7 @@ machine.node.dc1.test-domain. 0 IN TXT "consul-network-segment=" #### PTR queries -Responses to pointer record (PTR) queries, such as `.in-addr.arpa.`, always use the [primary domain](/consul/docs/reference/agent#domain) and not the alternative domain. +Responses to pointer record (PTR) queries, such as `.in-addr.arpa.`, always use the [primary domain](/consul/docs/reference/agent/configuration-file/general#domain) and not the alternative domain. ### Caching diff --git a/website/content/docs/discover/service/static.mdx b/website/content/docs/discover/service/static.mdx index 1b0952aec73..55d1a31ed1f 100644 --- a/website/content/docs/discover/service/static.mdx +++ b/website/content/docs/discover/service/static.mdx @@ -21,7 +21,7 @@ All versions of Consul support DNS lookup features. If ACLs are enabled, you must present a token linked with the necessary policies. We recommend using a separate token in production deployments for querying the DNS. By default, Consul agents resolve DNS requests using the preconfigured tokens in order of precedence: -1. The agent's [`default` token](/consul/docs/reference/agent#acl_tokens_default) +1. The agent's [`default` token](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_default) 1. The built-in [`anonymous` token](/consul/docs/secure/acl/token#built-in-tokens). The following table describes the available DNS lookups and required policies when ACLs are enabled: diff --git a/website/content/docs/east-west/mesh-gateway/enable.mdx b/website/content/docs/east-west/mesh-gateway/enable.mdx index cf577eb8bbe..c16fb9fdecb 100644 --- a/website/content/docs/east-west/mesh-gateway/enable.mdx +++ b/website/content/docs/east-west/mesh-gateway/enable.mdx @@ -126,7 +126,7 @@ connect { } ``` -The [`retry_join_wan`](/consul/docs/reference/agent#retry_join_wan) addresses are +The [`retry_join_wan`](/consul/docs/reference/agent/configuration-file/join#retry_join_wan) addresses are only used for the [traditional federation process](/consul/docs/east-west/wan-federation#traditional-wan-federation). They must be omitted when federating Consul servers via gateways. @@ -194,7 +194,7 @@ If the primary gateways are upgraded, and their previous instances are decommiss before the updates are propagated, then the primary datacenter will become unreachable. To safely upgrade primary gateways, we recommend that you apply one of the following policies: -- Avoid decommissioning primary gateway IP addresses. This is because the [primary_gateways](/consul/docs/reference/agent#primary_gateways) addresses configured on the secondary servers act as a fallback mechanism for re-establishing connectivity to the primary. +- Avoid decommissioning primary gateway IP addresses. This is because the [primary_gateways](/consul/docs/reference/agent/configuration-file/general#primary_gateways) addresses configured on the secondary servers act as a fallback mechanism for re-establishing connectivity to the primary. - Verify that addresses of the new mesh gateways in the primary were propagated to the secondary datacenters before decommissioning the old mesh gateways in the primary. diff --git a/website/content/docs/east-west/wan-federation/k8s-vm.mdx b/website/content/docs/east-west/wan-federation/k8s-vm.mdx index 8eee10f3918..5467dbee6c7 100644 --- a/website/content/docs/east-west/wan-federation/k8s-vm.mdx +++ b/website/content/docs/east-west/wan-federation/k8s-vm.mdx @@ -15,7 +15,7 @@ This page describes how to federate Consul clusters separately deployed in VM an for more information, including [Kubernetes networking requirements](/consul/docs/east-west/wan-federation#network-requirements). Consul datacenters running on non-kubernetes platforms like VMs or bare metal can -be federated with Kubernetes datacenters. +be federated with Kubernetes datacenters. ## Kubernetes as the Primary @@ -96,7 +96,7 @@ The following sections detail how to export this data. ==> Saved dc1-client-consul-0-key.pem ``` - Or use the [auto_encrypt](/consul/docs/reference/agent#auto_encrypt) feature. + Or use the [auto_encrypt](/consul/docs/reference/agent/configuration-file/encryption#auto_encrypt) feature. ### Mesh Gateway Addresses @@ -400,4 +400,4 @@ to allow traffic between datacenters. ## Next Steps In both cases (Kubernetes as primary or secondary), after installation, follow the [Verifying Federation](/consul/docs/east-west/wan-federation/k8s#verifying-federation) -section to verify that federation is working as expected. \ No newline at end of file +section to verify that federation is working as expected. diff --git a/website/content/docs/east-west/wan-federation/k8s.mdx b/website/content/docs/east-west/wan-federation/k8s.mdx index 18e3701cdd4..6b4fbad32ef 100644 --- a/website/content/docs/east-west/wan-federation/k8s.mdx +++ b/website/content/docs/east-west/wan-federation/k8s.mdx @@ -210,8 +210,8 @@ The automatically generated federation secret contains: - **Consul server config** - This is a JSON snippet that must be used as part of the server config for secondary datacenters. It sets: - - [`primary_datacenter`](/consul/docs/reference/agent#primary_datacenter) to the name of the primary datacenter. - - [`primary_gateways`](/consul/docs/reference/agent#primary_gateways) to an array of IPs or hostnames for the mesh gateways in the primary datacenter. These are the addresses that Consul servers in secondary clusters will use to communicate with the primary datacenter. + - [`primary_datacenter`](/consul/docs/reference/agent/configuration-file/general#primary_datacenter) to the name of the primary datacenter. + - [`primary_gateways`](/consul/docs/reference/agent/configuration-file/general#primary_gateways) to an array of IPs or hostnames for the mesh gateways in the primary datacenter. These are the addresses that Consul servers in secondary clusters will use to communicate with the primary datacenter. Even if there are multiple secondary datacenters, only the primary gateways need to be configured. Upon first connection with a primary datacenter, the addresses for other secondary datacenters will be discovered. @@ -375,4 +375,4 @@ With your Kubernetes clusters federated, complete the [Secure and Route Service You can also read our in-depth documentation on [Consul Service Mesh In Kubernetes](/consul/docs/k8s/connect). -If you are still considering a move to Kubernetes, or to Consul on Kubernetes specifically, our [Migrate to Microservices with Consul Service Mesh on Kubernetes](/consul/tutorials/microservices?utm_source=docs) collection uses an example application written by a fictional company to illustrate why and how organizations can migrate from monolith to microservices using Consul service mesh on Kubernetes. The case study in this collection should provide information valuable for understanding how to develop services that leverage Consul during any stage of your microservices journey. \ No newline at end of file +If you are still considering a move to Kubernetes, or to Consul on Kubernetes specifically, our [Migrate to Microservices with Consul Service Mesh on Kubernetes](/consul/tutorials/microservices?utm_source=docs) collection uses an example application written by a fictional company to illustrate why and how organizations can migrate from monolith to microservices using Consul service mesh on Kubernetes. The case study in this collection should provide information valuable for understanding how to develop services that leverage Consul during any stage of your microservices journey. diff --git a/website/content/docs/east-west/wan-federation/vms.mdx b/website/content/docs/east-west/wan-federation/vms.mdx index 41f1a888c4d..c0114429fdb 100644 --- a/website/content/docs/east-west/wan-federation/vms.mdx +++ b/website/content/docs/east-west/wan-federation/vms.mdx @@ -19,16 +19,16 @@ One of the key features of Consul is its support for multiple datacenters. The a ## Network requirements -There are a few networking requirements that must be satisfied for WAN federation to work. +There are a few networking requirements that must be satisfied for WAN federation to work. -- All server nodes must be able to talk to each other. Otherwise, the gossip protocol and RPC forwarding will not work. -- If you need to use service discovery across datacenters, the network must be able to route traffic between IP addresses across regions. +- All server nodes must be able to talk to each other. Otherwise, the gossip protocol and RPC forwarding will not work. +- If you need to use service discovery across datacenters, the network must be able to route traffic between IP addresses across regions. -Usually, this means that all datacenters must be connected using a VPN or other tunneling mechanism. Consul does not handle VPN or NAT traversal for you. For RPC forwarding to work, the bind address must be accessible from remote nodes. +Usually, this means that all datacenters must be connected using a VPN or other tunneling mechanism. Consul does not handle VPN or NAT traversal for you. For RPC forwarding to work, the bind address must be accessible from remote nodes. ## Setup two datacenters -To get started, follow the [Deployment guide](/consul/tutorials/production-deploy/deployment-guide) to start each datacenter. If ACLs are enabled or you are using Consul Enterprise, you must set the [`primary_datacenter`](/consul/docs/reference/agent#primary_datacenter) in the server's configuration in both datacenters. +To get started, follow the [Deployment guide](/consul/tutorials/production-deploy/deployment-guide) to start each datacenter. If ACLs are enabled or you are using Consul Enterprise, you must set the [`primary_datacenter`](/consul/docs/reference/agent/configuration-file/general#primary_datacenter) in the server's configuration in both datacenters. After bootstrapping, you should have two datacenters, `dc1` and `dc2`. Note that datacenter names are opaque to Consul. They are labels that help human operators reason about the Consul datacenters. @@ -57,7 +57,7 @@ consul-server-1.dc2 172.18.0.7:8302 alive server 1.20.2 2 dc2 def The command provides a list of all known members in the WAN gossip pool. In this case, the servers are not yet connected so each datacenter is only aware of local servers. - + The `consul members -wan` command only contains server nodes. Client nodes send requests to the local datacenter server, so they do not participate in WAN gossip. Local servers forward client requests to a server in the target datacenter as necessary. @@ -132,7 +132,7 @@ During the joining procedure, Consul records the following lines on each server ## Join the servers at startup -In a production environment, we recommend you setup WAN federation in Consul's configuration files. This way, Consul applies the configuration immediately at startup and it is persisted across reboots. Consul provides the [`retry_join_wan`](/consul/docs/reference/agent#retry_join_wan) parameter to configure the addresses of the other servers to WAN join at startup. +In a production environment, we recommend you setup WAN federation in Consul's configuration files. This way, Consul applies the configuration immediately at startup and it is persisted across reboots. Consul provides the [`retry_join_wan`](/consul/docs/reference/agent/configuration-file/join#retry_join_wan) parameter to configure the addresses of the other servers to WAN join at startup. @@ -241,11 +241,11 @@ Configuring `serf_wan`, `advertise_addr_wan` and `translate_wan_addrs` can lead The most likely cause of these errors is that `bind_addr` is set to a private address preventing the RPC server from accepting connections across the WAN. Setting `bind_addr` to a public address (or one that can be routed across the WAN) will resolve this issue. Be aware that exposing the RPC server on a public port should only be done **after** firewall rules have been established. -The [`translate_wan_addrs`](/consul/docs/reference/agent#translate_wan_addrs) configuration provides a basic address rewriting capability. +The [`translate_wan_addrs`](/consul/docs/reference/agent/configuration-file/general#translate_wan_addrs) configuration provides a basic address rewriting capability. ## Data replication -In general, data is not replicated between different Consul datacenters. When a request is made for a resource in another datacenter, the local Consul servers forward an RPC request to the remote Consul servers for that resource and return the results. If the remote datacenter is not available, then those resources will also not be available. However, this will not otherwise affect the local datacenter. +In general, data is not replicated between different Consul datacenters. When a request is made for a resource in another datacenter, the local Consul servers forward an RPC request to the remote Consul servers for that resource and return the results. If the remote datacenter is not available, then those resources will also not be available. However, this will not otherwise affect the local datacenter. There are some special situations where a limited subset of data can be replicated, such as with Consul's built-in ACL replication capability, or external tools like [consul-replicate](https://github.com/hashicorp/consul-replicate/). diff --git a/website/content/docs/enterprise/license/index.mdx b/website/content/docs/enterprise/license/index.mdx index cfaa1d9d767..c7cc8443cc3 100644 --- a/website/content/docs/enterprise/license/index.mdx +++ b/website/content/docs/enterprise/license/index.mdx @@ -30,7 +30,7 @@ Virtual agents are nodes that Consul service mesh services run on. HashiCorp use ### Applying a License For Consul Enterprise 1.10.0 or greater, a license must be available at the time the agent starts. -For server agents this means that they must either have the [`license_path`](/consul/docs/reference/agent#license_path) +For server agents this means that they must either have the [`license_path`](/consul/docs/reference/agent/configuration-file/general#license_path) configuration set or have a license configured in the servers environment with the `CONSUL_LICENSE` or `CONSUL_LICENSE_PATH` environment variables. Both the configuration item and the `CONSUL_LICENSE_PATH` environment variable point to a file containing the license whereas the `CONSUL_LICENSE` environment @@ -61,8 +61,8 @@ When a client agent starts without a license in its configuration or environment license from the servers via RPCs. That RPC always requires a valid non-anonymous ACL token to authorize the request but the token doesn't need any particular permissions. As the license is required before the client actually joins the cluster, where to make those RPC requests to is inferred from the -[`retry_join`](/consul/docs/reference/agent#retry_join) configuration. If `retry_join` is unset or no -[`agent` token](/consul/docs/reference/agent#acl_tokens_agent) is set then the client agent will immediately shut itself down. +[`retry_join`](/consul/docs/reference/agent/configuration-file/join#retry_join) configuration. If `retry_join` is unset or no +[`agent` token](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_agent) is set then the client agent will immediately shut itself down. If all preliminary checks pass the client agent will attempt to reach out to any server on its RPC port to request the license. These requests will be retried for up to 5 minutes and if it is unable to retrieve a diff --git a/website/content/docs/fundamentals/interface/ui.mdx b/website/content/docs/fundamentals/interface/ui.mdx index 9d267aeef26..05ed12ffd19 100644 --- a/website/content/docs/fundamentals/interface/ui.mdx +++ b/website/content/docs/fundamentals/interface/ui.mdx @@ -35,10 +35,10 @@ When you run `consul agent -dev` to start Consul in development mode, the web UI The web UI is not enabled by default. Agents that are run without the `-dev` flag must explicitly enable the web UI. -To enable the web UI, you can: +To enable the web UI, you can: - Modify the agent start-up command to include the [`-ui` flag](/consul/commands/agent#ui-options). -- Modify the agent configuration file to include the [`ui_config.enabled = true`](/consul/docs/reference/agent#ui_config_enabled) attribute. +- Modify the agent configuration file to include the [`ui_config.enabled = true`](/consul/docs/reference/agent/configuration-file/ui#ui_config_enabled) attribute. @@ -145,4 +145,4 @@ With ACLs enabled and proper permissions set, you can click on the ACL-related p Click the **Settings** option from the top navigation bar. -You can select whether or not to enable **Blocking Queries**. This allows the UI to update in real time without manually having to refresh the page. This setting is enabled by default but you may want to disable it on large datacenters for better performance. \ No newline at end of file +You can select whether or not to enable **Blocking Queries**. This allows the UI to update in real time without manually having to refresh the page. This setting is enabled by default but you may want to disable it on large datacenters for better performance. diff --git a/website/content/docs/manage-traffic/route-local.mdx b/website/content/docs/manage-traffic/route-local.mdx index db2940b84aa..ef8498c46b5 100644 --- a/website/content/docs/manage-traffic/route-local.mdx +++ b/website/content/docs/manage-traffic/route-local.mdx @@ -31,19 +31,19 @@ For networks deployed to virtual machines, complete the following steps to route If you deployed Consul to a Kubernetes or ECS environment using `consul-k8s` or `consul-ecs`, service instance locality information is inherited from the host machine. As a result, you do not need to specify the regions and zones on containerized platforms unless you are implementing a custom deployment. -On Kubernetes, Consul automatically populates geographic information about service instances using the `topology.kubernetes.io/region` and `topology.kubernetes.io/zone` labels from the Kubernetes nodes. On AWS ECS, Consul uses the `AWS_REGION` environment variable and `AvailabilityZone` attribute of the ECS task meta. +On Kubernetes, Consul automatically populates geographic information about service instances using the `topology.kubernetes.io/region` and `topology.kubernetes.io/zone` labels from the Kubernetes nodes. On AWS ECS, Consul uses the `AWS_REGION` environment variable and `AvailabilityZone` attribute of the ECS task meta. ### Requirements -You should only enable locality-aware routing when each set of external upstream instances within the same zone and region have enough capacity to handle requests from downstream service instances in their respective zones. Locality-aware routing is an advanced feature that may adversely impact service capacity if used incorrectly. When enabled, Consul routes all traffic to the nearest set of service instances and only fails over when no healthy instances are available in the nearest set. +You should only enable locality-aware routing when each set of external upstream instances within the same zone and region have enough capacity to handle requests from downstream service instances in their respective zones. Locality-aware routing is an advanced feature that may adversely impact service capacity if used incorrectly. When enabled, Consul routes all traffic to the nearest set of service instances and only fails over when no healthy instances are available in the nearest set. ## Specify the locality of your Consul agents The `locality` configuration on a Consul client applies to all services registered to the client. -1. Configure the `locality` block in your Consul client agent configuration files. The `locality` block is a map containing the `region` and `zone` parameters. +1. Configure the `locality` block in your Consul client agent configuration files. The `locality` block is a map containing the `region` and `zone` parameters. - The parameters should match the values for regions and zones defined in your network. Refer to [`locality`](/consul/docs/reference/agent#locality) in the agent configuration reference for additional information. + The parameters should match the values for regions and zones defined in your network. Refer to [`locality`](/consul/docs/reference/agent/configuration-file/service-mesh#locality) in the agent configuration reference for additional information. 1. Start or restart the agent to apply the configuration. Refer to [Starting a Consul agent](/consul/docs/agent#starting-the-consul-agent) for instructions. @@ -58,7 +58,7 @@ locality = { ## Specify the localities of your service instances (optional) -This step is optional in most scenarios. Refer to [Workflow](#workflow) for additional information. +This step is optional in most scenarios. Refer to [Workflow](#workflow) for additional information. 1. Configure the `locality` block in your service definition for both downstream (client) and upstream services. The `locality` block is a map containing the `region` and `zone` parameters. When you start a proxy for the service, Consul passes the locality to the proxy so that it can route traffic accordingly. @@ -69,7 +69,7 @@ Register or re-register the service to apply the configuration. Refer to [Regist In the following example, the `web` service is available in the `us-west-1` region and `us-west-1a` zone on AWS: -```hcl +```hcl service { id = "web" locality = { @@ -171,7 +171,7 @@ Apply the configuration by either running the [`consul config write` CLI command - + ```hcl Kind = "service-resolver" Name = "api" @@ -181,10 +181,10 @@ Apply the configuration by either running the [`consul config write` CLI command ``` - + - + ```json { "kind": "service-resolver", @@ -198,7 +198,7 @@ Apply the configuration by either running the [`consul config write` CLI command - + ```yaml apiversion: consul.hashicorp.com/v1alpha1 kind: ServiceResolver @@ -208,7 +208,7 @@ Apply the configuration by either running the [`consul config write` CLI command prioritizeByLocality: mode: failover ``` - + @@ -217,32 +217,32 @@ Apply the configuration by either running the [`consul config write` CLI command - + ```shell-session $ consul config write api-resolver.hcl ``` - + - + - + ```shell-session $ kubectl apply -f api-resolver.yaml ``` - + - + ```shell-session $ curl --request PUT --data @api-resolver.hcl http://127.0.0.1:8500/v1/config ``` - + ### Configure locality on Kubernetes test clusters explicitly -You can explicitly configure locality for each Kubernetes node so that you can test locality-aware routing with a local Kubernetes cluster or in an environment where `topology.kubernetes.io` labels are not set. +You can explicitly configure locality for each Kubernetes node so that you can test locality-aware routing with a local Kubernetes cluster or in an environment where `topology.kubernetes.io` labels are not set. Run the `kubectl label node` command and specify the locality as arguments. The following example specifies the `us-east-1` region and `us-east-1a` zone for the node: diff --git a/website/content/docs/manage/rate-limit/initialize.mdx b/website/content/docs/manage/rate-limit/initialize.mdx index 80e4aa6db71..8b163d56cf4 100644 --- a/website/content/docs/manage/rate-limit/initialize.mdx +++ b/website/content/docs/manage/rate-limit/initialize.mdx @@ -10,22 +10,22 @@ Because each network has different needs and application, you need to find out w 1. In the agent configuration file, specify a global rate limit with arbitrary values based on the following conditions: - - Environment where Consul servers are running + - Environment where Consul servers are running - Number of servers and the projected load - Existing metrics expressing requests per second -1. Set the [`limits.request_limits.mode`](/consul/docs/reference/agent#mode-1) parameter in the agent configuration to `permissive`. In the following example, the configuration allows up to 1000 reads and 500 writes per second for each Consul agent: +1. Set the [`limits.request_limits.mode`](/consul/docs/reference/agent/configuration-file/general#mode) parameter in the agent configuration to `permissive`. In the following example, the configuration allows up to 1000 reads and 500 writes per second for each Consul agent: ```hcl - request_limits { + request_limits { mode = "permissive" read_rate = 1000.0 write_rate = 500.0 } - ``` -1. Observe the logs and metrics for your application's typical cycle, such as a 24 hour period. Refer to [Monitor traffic rate limit data](/consul/docs/manage/rate-limit/monitor) for additional information. Call the [`/agent/metrics`](/consul/api-docs/agent#view-metrics) HTTP API endpoint and check the data for the following metrics: + ``` +1. Observe the logs and metrics for your application's typical cycle, such as a 24 hour period. Refer to [Monitor traffic rate limit data](/consul/docs/manage/rate-limit/monitor) for additional information. Call the [`/agent/metrics`](/consul/api-docs/agent#view-metrics) HTTP API endpoint and check the data for the following metrics: - `rpc.rate_limit.exceeded` with value `global/read` for label `limit_type` - `rpc.rate_limit.exceeded` with value `global/write` for label `limit_type` -1. If the limits are not reached, set the `mode` configuration to `enforcing`. Otherwise, continue to adjust and iterate until you find your network's unique limits. \ No newline at end of file +1. If the limits are not reached, set the `mode` configuration to `enforcing`. Otherwise, continue to adjust and iterate until you find your network's unique limits. diff --git a/website/content/docs/manage/rate-limit/source.mdx b/website/content/docs/manage/rate-limit/source.mdx index 9f33759787a..59d4fc6417a 100644 --- a/website/content/docs/manage/rate-limit/source.mdx +++ b/website/content/docs/manage/rate-limit/source.mdx @@ -69,4 +69,4 @@ $ kubectl apply control-plane-request-limit.yaml ## Disable request rate limits -Set the [limits.request_limits.mode](/consul/docs/reference/agent#mode-1) in the agent configuration to `disabled` to allow services to exceed the specified read and write requests limits. The `disabled` mode applies to all request rate limits, even limits specified in the [control plane request limits configuration entry](/consul/docs/reference/config-entry/control-plane-request-limit). Note that any other mode specified in the agent configuration only applies to global traffic rate limits. +Set the [limits.request_limits.mode](/consul/docs/reference/agent/configuration-file/general#mode) in the agent configuration to `disabled` to allow services to exceed the specified read and write requests limits. The `disabled` mode applies to all request rate limits, even limits specified in the [control plane request limits configuration entry](/consul/docs/reference/config-entry/control-plane-request-limit). Note that any other mode specified in the agent configuration only applies to global traffic rate limits. diff --git a/website/content/docs/manage/scale/index.mdx b/website/content/docs/manage/scale/index.mdx index 2666af27b41..fb563bd87eb 100644 --- a/website/content/docs/manage/scale/index.mdx +++ b/website/content/docs/manage/scale/index.mdx @@ -65,7 +65,7 @@ For most use cases, a limit of 5,000 agents is appropriate. When the `consul.ser In Kubernetes, even though it is possible to deploy Consul agents inside pods alongside services running in the same pod, this unsupported deployment pattern has known performance issues at scale. At large volumes, pod registration and deregistration in Kubernetes causes gossip instability that can lead to cascading failures as services are marked unhealthy, resulting in further cluster churn. -In Consul v1.14 and higher, Consul on Kubernetes does not need to run client agents on every node in a cluster for service discovery and service mesh. This deployment configuration lowers Consul’s resource usage in the data plane, but requires additional resources in the control plane to process [xDS resources](/consul/docs/reference/agent#xds-server-parameters). To learn more, refer to [simplified service mesh with Consul Dataplane](/consul/docs/architecture/control-plane/dataplane). +In Consul v1.14 and higher, Consul on Kubernetes does not need to run client agents on every node in a cluster for service discovery and service mesh. This deployment configuration lowers Consul’s resource usage in the data plane, but requires additional resources in the control plane to process [xDS resources](/consul/docs/reference/agent/configuration-file/xds#xds-server-parameters). To learn more, refer to [simplified service mesh with Consul Dataplane](/consul/docs/architecture/control-plane/dataplane). **If you use Kubernetes and Consul as a backend for Vault**: Use Vault’s integrated storage backend instead of Consul. A runtime dependency conflict prevents Consul dataplanes from being compatible with Vault. If you need to use Consul v1.14 and higher as a backend for Vault in your Kubernetes deployment, create a separate Consul datacenter that is not federated or peered to your other Consul servers. You can size this datacenter according to your needs and use it exclusively for backend storage for Vault. @@ -126,7 +126,7 @@ We strongly recommend using [stale consistency mode for DNS lookups](/consul/api We also recommend that you do not configure [`dns_config.max_stale` to limit the staleness of DNS responses](/consul/api-docs/features/consistency#limiting-staleness-advanced-usage), as it may result in a prolonged outage if your Consul servers become overloaded. If bounded result consistency is required by a service, consider modifying the service to use consistent service discovery HTTP API queries instead of DNS lookups. -Avoid using [`dns_config.use_cache`](/consul/docs/reference/agent#dns_use_cache) when operating Consul at scale. Because the Consul agent cache allocates memory for each requested route and each allocation can live up to 3 days, severe memory issues may occur. To implement DNS caching, we instead recommend that you [configure TTLs for services and nodes](/consul/tutorials/networking/dns-caching#ttl) to enable the DNS client to cache responses from Consul. +Avoid using [`dns_config.use_cache`](/consul/docs/reference/agent/configuration-file/dns#dns_use_cache) when operating Consul at scale. Because the Consul agent cache allocates memory for each requested route and each allocation can live up to 3 days, severe memory issues may occur. To implement DNS caching, we instead recommend that you [configure TTLs for services and nodes](/consul/tutorials/networking/dns-caching#ttl) to enable the DNS client to cache responses from Consul. #### HTTP API @@ -210,7 +210,7 @@ When the leader takes snapshot B at index 199, it truncates the logs that accumu Because the new server restored snapshot A, the new server has a current index of 99. It requests logs 100 to 150 because index 150 was the current index when it started the replication restore process. At this point, the leader recognizes that it only has logs 200 and higher, and does not have logs for indexes 100 to 150. The leader determines that the new server’s state is stale and starts the process over by sending the new server the latest snapshot, snapshot B. -Consul keeps a configurable number of [Raft trailing logs](/consul/docs/reference/agent#raft_trailing_logs) to prevent the snapshot install loop from repeating. The trailing logs are the last logs that went into the snapshot, and the new server can more easily catch up to the current state using these logs. The default Raft trailing logs configuration value is suitable for most deployments. +Consul keeps a configurable number of [Raft trailing logs](/consul/docs/reference/agent/configuration-file/raft#raft_trailing_logs) to prevent the snapshot install loop from repeating. The trailing logs are the last logs that went into the snapshot, and the new server can more easily catch up to the current state using these logs. The default Raft trailing logs configuration value is suitable for most deployments. In Consul v1.10 and higher, operators can try to prevent a snapshot install loop by monitoring and comparing Consul servers’ `consul.raft.rpc.installSnapshot` and `consul.raft.leader.oldestLogAge` timing metrics. Monitor these metrics for the following situations: @@ -257,7 +257,7 @@ Consul ESM enables health checks and monitoring for external services. When usin Because Consul’s service mesh uses service discovery subsystems, service mesh performance is also optimized by deploying multiple small clusters with consistent numbers of service instances and watches. Service mesh performance is influenced by the following additional factors: - The [transparent proxy](/consul/docs/connect/transparent-proxy) feature causes client agents to listen for service instance updates across all services instead of a subset. To prevent performance issues, we recommend that you do not use the permissive intention, `default: allow`, with the transparent proxy feature. When combined, every service instance update propagates to every proxy, which causes additional server load. -- When you use the [built-in service mesh CA provider](/consul/docs/secure-mesh/certificate/built-in#built-in-ca), Consul leaders are responsible for signing certificates used for mTLS across the service mesh. The impact on CPU utilization depends on the total number of service instances and configured certificate TTLs. You can use the [CA provider configuration options](/consul/docs/reference/agent#common-ca-config-options) to control the number of requests a server processes. We recommend adjusting [`csr_max_concurrent`](/consul/docs/reference/agent#ca_csr_max_concurrent) and [`csr_max_per_second`](/consul/docs/reference/agent#ca_csr_max_concurrent) to suit your environment. +- When you use the [built-in service mesh CA provider](/consul/docs/secure-mesh/certificate/built-in#built-in-ca), Consul leaders are responsible for signing certificates used for mTLS across the service mesh. The impact on CPU utilization depends on the total number of service instances and configured certificate TTLs. You can use the [CA provider configuration options](/consul/docs/reference/agent/configuration-file/service-mesh#common-ca-config-options) to control the number of requests a server processes. We recommend adjusting [`csr_max_concurrent`](/consul/docs/reference/agent/configuration-file/service-mesh#ca_csr_max_concurrent) and [`csr_max_per_second`](/consul/docs/reference/agent/configuration-file/service-mesh#ca_csr_max_per_second) to suit your environment. ### K/V store diff --git a/website/content/docs/monitor/log/audit.mdx b/website/content/docs/monitor/log/audit.mdx index 23c394a1031..4be1d566456 100644 --- a/website/content/docs/monitor/log/audit.mdx +++ b/website/content/docs/monitor/log/audit.mdx @@ -17,13 +17,13 @@ With Consul Enterprise v1.8.0+, audit logging can be used to capture a clear and Audit logging enables security and compliance teams within an organization to get greater insight into Consul access and usage patterns. -Complete the [Capture Consul Events with Audit Logging](/consul/tutorials/datacenter-operations/audit-logging) tutorial to learn more about Consul's audit logging functionality, +Complete the [Capture Consul Events with Audit Logging](/consul/tutorials/datacenter-operations/audit-logging) tutorial to learn more about Consul's audit logging functionality, -For detailed configuration information on configuring the Consul Enterprise's audit logging, review the Consul [Audit Log](/consul/docs/reference/agent#audit) documentation. +For detailed configuration information on configuring the Consul Enterprise's audit logging, review the Consul [Audit Log](/consul/docs/reference/agent/configuration-file/general#audit) documentation. ## Example Configuration -Audit logging must be enabled on every agent in order to accurately capture all operations performed through the HTTP API. To enable logging, add the [`audit`](/consul/docs/reference/agent#audit) stanza to the agent's configuration. +Audit logging must be enabled on every agent in order to accurately capture all operations performed through the HTTP API. To enable logging, add the [`audit`](/consul/docs/reference/agent/configuration-file/general#audit) stanza to the agent's configuration. -> **Note**: Consul only logs operations which are initiated via the HTTP API. The audit log does not record operations that take place over the internal RPC communication channel used for agent communication. diff --git a/website/content/docs/monitor/telemetry/agent.mdx b/website/content/docs/monitor/telemetry/agent.mdx index 8d525aaf8ad..12ea59ed235 100644 --- a/website/content/docs/monitor/telemetry/agent.mdx +++ b/website/content/docs/monitor/telemetry/agent.mdx @@ -29,9 +29,9 @@ Metrics tutorial](/consul/tutorials/day-2-operations/monitor-datacenter-health?u By default, all metric names of gauge type are prefixed with the hostname of the consul agent, e.g., `consul.hostname.server.isLeader`. To disable prefixing the hostname, set -`telemetry.disable_hostname=true` in the [agent configuration](/consul/docs/reference/agent#telemetry). +`telemetry.disable_hostname=true` in the [agent configuration](/consul/docs/reference/agent/configuration-file/telemetry). -Additionally, if the [`telemetry` configuration options](/consul/docs/reference/agent#telemetry) +Additionally, if the [`telemetry` configuration options](/consul/docs/reference/agent/configuration-file/telemetry) are provided, the telemetry information will be streamed to a [statsite](http://github.com/armon/statsite) or [statsd](http://github.com/etsy/statsd) server where it can be aggregated and flushed to Graphite or any other metrics store. @@ -218,7 +218,7 @@ Under these conditions, a follower after a restart may be unable to catch up on replication and become a voter again since it takes longer to restore from disk or the leader than the leader takes to write a new snapshot and truncate its logs. Servers retain -[`raft_trailing_logs`](/consul/docs/reference/agent#raft_trailing_logs) (default +[`raft_trailing_logs`](/consul/docs/reference/agent/configuration-file/raft#raft_trailing_logs) (default `10240`) log entries even if their snapshot was more recent. On a leader processing 500 commits/second, that is only about 20 seconds worth of logs. Assuming the leader is able to write out a snapshot and truncate the logs in @@ -243,7 +243,7 @@ repeatedly as well as reduce the fault tolerance and serving capacity of the cluster. Since Consul 1.5.3 -[`raft_trailing_logs`](/consul/docs/reference/agent#raft_trailing_logs) has been +[`raft_trailing_logs`](/consul/docs/reference/agent/configuration-file/raft#raft_trailing_logs) has been configurable. Increasing it allows the leader to retain more logs and give followers more time to restore and catch up. The tradeoff is potentially slower appends which eventually might affect write throughput and latency @@ -254,7 +254,7 @@ mean loosing cluster availability and needing to recover the cluster from a loss of quorum. Since Consul 1.10.0 -[`raft_trailing_logs`](/consul/docs/reference/agent#raft_trailing_logs) is now +[`raft_trailing_logs`](/consul/docs/reference/agent/configuration-file/raft#raft_trailing_logs) is now reloadable with `consul reload` or `SIGHUP` allowing operators to increase this without the leader restarting or loosing leadership allowing the cluster to be recovered gracefully. @@ -319,7 +319,7 @@ This metric should be monitored to ensure that the license doesn't expire to pre | Metric Name | Description | Unit | Type | | :-------------------------------- | :--------------------------------------------------------------- | :---- | :---- | -| `consul.raft.boltdb.freelistBytes` | Represents the number of bytes necessary to encode the freelist metadata. When [`raft_logstore.boltdb.no_freelist_sync`](/consul/docs/reference/agent#raft_logstore_boltdb_no_freelist_sync) is set to `false` these metadata bytes must also be written to disk for each committed log. | bytes | gauge | +| `consul.raft.boltdb.freelistBytes` | Represents the number of bytes necessary to encode the freelist metadata. When [`raft_logstore.boltdb.no_freelist_sync`](/consul/docs/reference/agent/configuration-file/raft#raft_logstore_boltdb_no_freelist_sync) is set to `false` these metadata bytes must also be written to disk for each committed log. | bytes | gauge | | `consul.raft.boltdb.logsPerBatch` | Measures the number of logs being written per batch to the db. | logs | sample | | `consul.raft.boltdb.storeLogs` | Measures the amount of time spent writing logs to the db. | ms | timer | | `consul.raft.boltdb.writeCapacity` | Theoretical write capacity in terms of the number of logs that can be written per second. Each sample outputs what the capacity would be if future batched log write operations were similar to this one. This similarity encompasses 4 things: batch size, byte size, disk performance and boltdb performance. While none of these will be static and its highly likely individual samples of this metric will vary, aggregating this metric over a larger time window should provide a decent picture into how this BoltDB store can perform | logs/second | sample | @@ -362,7 +362,7 @@ indicator of an actual issue, this metric can be used to diagnose why the `consu is high. If Bolt DB log storage performance becomes an issue and is caused by free list management then setting -[`raft_logstore.boltdb.no_freelist_sync`](/consul/docs/reference/agent#raft_logstore_boltdb_no_freelist_sync) to `true` in the server's configuration +[`raft_logstore.boltdb.no_freelist_sync`](/consul/docs/reference/agent/configuration-file/raft#raft_logstore_boltdb_no_freelist_sync) to `true` in the server's configuration may help to reduce disk IO and log storage operation times. Disabling free list syncing will however increase the startup time for a server as it must scan the raft.db file for free space instead of loading the already populated free list structure. diff --git a/website/content/docs/multi-tenant/network-segment/index.mdx b/website/content/docs/multi-tenant/network-segment/index.mdx index d9833ed7b33..2ef8f7260f1 100644 --- a/website/content/docs/multi-tenant/network-segment/index.mdx +++ b/website/content/docs/multi-tenant/network-segment/index.mdx @@ -7,18 +7,18 @@ description: >- # Network segments overview -Network segmentation is the practice of dividing a network into multiple segments or subnets that act as independent networks. This topic provides an overview of concepts related to operating Consul in a segmented network. +Network segmentation is the practice of dividing a network into multiple segments or subnets that act as independent networks. This topic provides an overview of concepts related to operating Consul in a segmented network. -This feature requires Consul Enterprise version 0.9.3 or later. +This feature requires Consul Enterprise version 0.9.3 or later. Refer to the [enterprise feature matrix](/consul/docs/enterprise#consul-enterprise-feature-availability) for additional information. ## Segmented networks -Consul requires full connectivity between all agents in a datacenter within a LAN gossip pool. In some environments, however, business policies enforced through network rules or firewalls prevent full connectivity between all agents. These environments are called _segmented networks_. Network segments are isolated LAN gossip pools that only require full connectivity between agent members on the same segment. +Consul requires full connectivity between all agents in a datacenter within a LAN gossip pool. In some environments, however, business policies enforced through network rules or firewalls prevent full connectivity between all agents. These environments are called _segmented networks_. Network segments are isolated LAN gossip pools that only require full connectivity between agent members on the same segment. To use Consul in a segmented network, you must define the segments in your server agent configuration and direct client agents to join one of the segments. The Consul network segment configuration should match the LAN gossip pool boundaries. The following diagram shows how a network may be segmented: @@ -26,15 +26,15 @@ To use Consul in a segmented network, you must define the segments in your serve ## Default network segment -By default, all Consul agents are part of a shared Serf LAN gossip pool, referred to as the `` network segment. Because all agents are within the same segment, full mesh connectivity within the datacenter is required. The following diagram shows the `` network segment: +By default, all Consul agents are part of a shared Serf LAN gossip pool, referred to as the `` network segment. Because all agents are within the same segment, full mesh connectivity within the datacenter is required. The following diagram shows the `` network segment: ![Consul datacenter default agent connectivity: one network segment](/img/network-segments/consul-network-segments-single.png) ## Segment membership -Server agents are members of all segments. The datacenter includes the `` segment, as well as additional segments defined in the `segments` server agent configuration option. Refer to the [`segments`](/consul/docs/reference/agent#segments) documentation for additional information. +Server agents are members of all segments. The datacenter includes the `` segment, as well as additional segments defined in the `segments` server agent configuration option. Refer to the [`segments`](/consul/docs/reference/agent/configuration-file/general#segments) documentation for additional information. -Each client agent can only be a member of one segment at a time. Client agents are members of the `` segment unless they are configured to join a different segment. +Each client agent can only be a member of one segment at a time. Client agents are members of the `` segment unless they are configured to join a different segment. For a client agent to join the Consul datacenter, it must connect to another agent (client or server) within its configured segment. Read the [Network Segments documentation](/consul/docs/multi-tenant/network-segment) to learn more about network segments. diff --git a/website/content/docs/multi-tenant/network-segment/vm.mdx b/website/content/docs/multi-tenant/network-segment/vm.mdx index facb4a705f5..8c2010572d7 100644 --- a/website/content/docs/multi-tenant/network-segment/vm.mdx +++ b/website/content/docs/multi-tenant/network-segment/vm.mdx @@ -14,7 +14,7 @@ This topic describes how to create Consul network segments so that services can ## Define segments in the server configuration -1. Add the `segments` block to your server configuration. Refer to the [`segments`](/consul/docs/reference/agent#segments) documentation for details about how to define the configuration. +1. Add the `segments` block to your server configuration. Refer to the [`segments`](/consul/docs/reference/agent/configuration-file/general#segments) documentation for details about how to define the configuration. In the following example, an `alpha` segment is configured to listen for traffic on port `8303` and a `beta` segment is configured to listen to traffic on port `8304`: @@ -85,7 +85,7 @@ Client agents can only be members of one segment at a time. You can direct clien $ consul agent -config-file client.hcl -join 10.20.10.11:8303 -segment alpha ``` -Alternatively, you can add the [`retry_join`](/consul/docs/reference/agent#retry_join) and [`segment`](/consul/docs/reference/agent#segment-1) parameters to your client agent configuration file: +Alternatively, you can add the [`retry_join`](/consul/docs/reference/agent/configuration-file/join#retry_join) and [`segment`](/consul/docs/reference/agent/configuration-file/general#segment-1) parameters to your client agent configuration file: ```hcl node_name = "consul-client" diff --git a/website/content/docs/north-south/ingress-gateway/external.mdx b/website/content/docs/north-south/ingress-gateway/external.mdx index 8bce27e2785..7afe0372655 100644 --- a/website/content/docs/north-south/ingress-gateway/external.mdx +++ b/website/content/docs/north-south/ingress-gateway/external.mdx @@ -23,8 +23,8 @@ The following process describes the general procedure for configuring ingress ga - The external service must implement Envoy's [gRPC secret discovery service (SDS) API](https://www.envoyproxy.io/docs/envoy/latest/configuration/security/secret). - You should have some familiarity with Envoy configuration and the SDS protocol. -- The [`connect.enabled` parameter](/consul/docs/reference/agent#connect) must be set to `true` for all server agents in the Consul datacenter. -- The [`ports.grpc` parameter](/consul/docs/reference/agent#connect) must be configured for all server agents in the Consul datacenter. +- The [`connect.enabled` parameter](/consul/docs/reference/agent/configuration-file/service-mesh#connect) must be set to `true` for all server agents in the Consul datacenter. +- The [`ports.grpc` parameter](/consul/docs/reference/agent/configuration-file/service-mesh#connect) must be configured for all server agents in the Consul datacenter. ### ACL requirements @@ -242,7 +242,7 @@ Listeners = [ ## Register the ingress gateway configuration entry -You can register the configuration entry using the [`consul config` command](/consul/commands/config) or by calling the [`/config` API endpoint](/consul/api-docs/config). +You can register the configuration entry using the [`consul config` command](/consul/commands/config) or by calling the [`/config` API endpoint](/consul/api-docs/config). The following example registers an ingress gateway configuration entry named `public-ingress-cfg.hcl` that is stored on the local system: @@ -250,4 +250,4 @@ The following example registers an ingress gateway configuration entry named `pu $ consul config write public-ingress-cfg.hcl ``` -The Envoy instance starts a listener on the port specified in the configuration entry and fetches the TLS certificate named from the SDS server. \ No newline at end of file +The Envoy instance starts a listener on the port specified in the configuration entry and fetches the TLS certificate named from the SDS server. diff --git a/website/content/docs/north-south/ingress-gateway/vm.mdx b/website/content/docs/north-south/ingress-gateway/vm.mdx index 2624d22f53c..e911f10d086 100644 --- a/website/content/docs/north-south/ingress-gateway/vm.mdx +++ b/website/content/docs/north-south/ingress-gateway/vm.mdx @@ -30,7 +30,7 @@ After specifying listeners and services in the ingress gateway configuration ent ## Requirements -- Service mesh must be enabled for all agents. Set the [`connect.enabled` parameter](/consul/docs/reference/agent#connect) to `true` to enable service mesh. +- Service mesh must be enabled for all agents. Set the [`connect.enabled` parameter](/consul/docs/reference/agent/configuration-file/service-mesh#connect) to `true` to enable service mesh. - The gRPC port must be configured for all server agents in the datacenter. Specify the gRPC port in the [`ports.grpc` parameter](/consul/docs/reference/agent/configuration-file/general#grpc_port). We recommend setting the port to `8502` to simplify configuration when ACLs are enabled. Refer to [ACL requirements](#acl-requirements) for additional information. - You must use Envoy for sidecar proxies in your service mesh. Refer to [Configure and deploy sidecar proxies](/consul/docs/connect/proxy/sidecar) for additional information. diff --git a/website/content/docs/observe/telemetry/vm.mdx b/website/content/docs/observe/telemetry/vm.mdx index 608c403f94f..d513fe54486 100644 --- a/website/content/docs/observe/telemetry/vm.mdx +++ b/website/content/docs/observe/telemetry/vm.mdx @@ -45,11 +45,11 @@ UI. If there are multiple clients with the UI enabled in a datacenter for redundancy these configurations must be added to all of them. We assume that the UI is already enabled by setting -[`ui_config.enabled`](/consul/docs/reference/agent#ui_config_enabled) to `true` in the +[`ui_config.enabled`](/consul/docs/reference/agent/configuration-file/ui#ui_config_enabled) to `true` in the agent's configuration file. To use the built-in Prometheus provider -[`ui_config.metrics_provider`](/consul/docs/reference/agent#ui_config_metrics_provider) +[`ui_config.metrics_provider`](/consul/docs/reference/agent/configuration-file/ui#ui_config_metrics_provider) must be set to `prometheus`. The UI must query the metrics provider through a proxy endpoint. This simplifies @@ -57,7 +57,7 @@ deployment where Prometheus is not exposed externally to UI user's browsers. To set this up, provide the URL that the _Consul agent_ should use to reach the Prometheus server in -[`ui_config.metrics_proxy.base_url`](/consul/docs/reference/agent#ui_config_metrics_proxy_base_url). +[`ui_config.metrics_proxy.base_url`](/consul/docs/reference/agent/configuration-file/ui#ui_config_metrics_proxy_base_url). For example in Kubernetes, the Prometheus helm chart by default installs a service named `prometheus-server` so each Consul agent can reach it on `http://prometheus-server` (using Kubernetes' DNS resolution). @@ -122,7 +122,7 @@ service-specific dashboard in an external tool like [Grafana](https://grafana.com) or a hosted provider. To configure this, you must provide a URL template in the [agent configuration -file](/consul/docs/reference/agent#ui_config_dashboard_url_templates) for all agents that +file](/consul/docs/reference/agent/configuration-file/ui#ui_config_dashboard_url_templates) for all agents that have the UI enabled. The template is essentially the URL to the external dashboard, but can have placeholder values which will be replaced with the service name, namespace and datacenter where appropriate to allow deep-linking @@ -657,12 +657,12 @@ ui_config { More than one JavaScript file may be specified in -[`metrics_provider_files`](/consul/docs/reference/agent#ui_config_metrics_provider_files) +[`metrics_provider_files`](/consul/docs/reference/agent/configuration-file/ui#ui_config_metrics_provider_files) and all will be served allowing flexibility if needed to include dependencies. Only one metrics provider can be configured and used at one time. The -[`metrics_provider_options_json`](/consul/docs/reference/agent#ui_config_metrics_provider_options_json) +[`metrics_provider_options_json`](/consul/docs/reference/agent/configuration-file/ui#ui_config_metrics_provider_options_json) field is an optional literal JSON object which is passed to the provider's `init` method at startup time. This allows configuring arbitrary parameters for the provider in config rather than hard coding them into the provider itself to @@ -671,7 +671,7 @@ make providers more reusable. The provider may fetch metrics directly from another source although in this case the agent will probably need to serve the correct CORS headers to prevent browsers from blocking these requests. These may be configured with -[`http_config.response_headers`](/consul/docs/reference/agent#response_headers). +[`http_config.response_headers`](/consul/docs/reference/agent/configuration-file/general#response_headers). Alternatively, the provider may choose to use the [built-in metrics proxy](#metrics-proxy) to avoid cross domain issues or to inject additional diff --git a/website/content/docs/reference/acl/policy.mdx b/website/content/docs/reference/acl/policy.mdx index 31089cd1929..412445ba672 100644 --- a/website/content/docs/reference/acl/policy.mdx +++ b/website/content/docs/reference/acl/policy.mdx @@ -104,7 +104,7 @@ Use the `policy` keyword and one of the following access levels to set a policy - `write`: Allows the resource to be read and modified. - `deny`: Denies read and write access to the resource. -The special `list` access level provides access to all keys with the specified resource label in the [Consul KV](/consul/commands/kv/). The `list` access level can only be used with the `key_prefix` resource. The [`acl.enable_key_list_policy`](/consul/docs/reference/agent#acl_enable_key_list_policy) setting must be set to `true`. +The special `list` access level provides access to all keys with the specified resource label in the [Consul KV](/consul/commands/kv/). The `list` access level can only be used with the `key_prefix` resource. The [`acl.enable_key_list_policy`](/consul/docs/reference/agent/configuration-file/acl#acl_enable_key_list_policy) setting must be set to `true`. ### Matching and Prefix Values @@ -246,7 +246,7 @@ operator = "read" ## Rule Scope The rules from all policies, including roles and service identities, linked with a token are combined to form that token's effective rule set. -Policy rules can be defined in either an `allowlist` or `denylist` mode, depending on the configuration of the [`acl_default_policy`](/consul/docs/reference/agent#acl_default_policy). +Policy rules can be defined in either an `allowlist` or `denylist` mode, depending on the configuration of the [`acl_default_policy`](/consul/docs/reference/agent/configuration-file/acl#acl_default_policy). If the default policy is configured to deny access to all resources, then you can specify `allowlist` in policy rules to explicitly allow access to resources. Conversely, if the default policy is configured to allow access to all resources, then you can specify `denylist` in policy rules to explicitly deny access to resources. diff --git a/website/content/docs/reference/acl/role.mdx b/website/content/docs/reference/acl/role.mdx index edfc866e1b6..b0abb09ad5d 100644 --- a/website/content/docs/reference/acl/role.mdx +++ b/website/content/docs/reference/acl/role.mdx @@ -72,7 +72,7 @@ Roles may contain the following attributes: You can specify a templated policy when configuring roles or linking tokens to policies. Templated policies enable you to quickly construct policies for common Consul use cases, rather than creating identical policies for each use cases. -Consul uses templated policies during the authorization process to automatically generate a policy for the use case specified. Consul links the generated policy to the role or token so that it will have permission for the specific use case. +Consul uses templated policies during the authorization process to automatically generate a policy for the use case specified. Consul links the generated policy to the role or token so that it will have permission for the specific use case. ### Templated policy specification @@ -308,7 +308,7 @@ node_prefix "" { You can specify a node identity when configuring roles or linking tokens to policies. _Node_ commonly refers to a Consul agent, but a node can also be a physical server, cloud instance, virtual machine, or container. -Node identities enable you to quickly construct policies for nodes, rather than manually creating identical polices for each node. They are used during the authorization process to automatically generate a policy for the node(s) specified. You can specify the token linked to the policy in the [`acl_tokens_agent`](/consul/docs/reference/agent#acl_tokens_agent) field when configuring the agent. +Node identities enable you to quickly construct policies for nodes, rather than manually creating identical polices for each node. They are used during the authorization process to automatically generate a policy for the node(s) specified. You can specify the token linked to the policy in the [`acl_tokens_agent`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_agent) field when configuring the agent. ### Node identity specification @@ -398,4 +398,4 @@ service_prefix "" { } ``` - \ No newline at end of file + diff --git a/website/content/docs/reference/acl/rule.mdx b/website/content/docs/reference/acl/rule.mdx index dfad455c0c5..fbe5ba94187 100644 --- a/website/content/docs/reference/acl/rule.mdx +++ b/website/content/docs/reference/acl/rule.mdx @@ -203,7 +203,7 @@ name that starts with `bar`. Since [Agent API](/consul/api-docs/agent) utility operations may be required before an agent is joined to a cluster, or during an outage of the Consul servers or ACL datacenter, a special token may be -configured with [`acl.tokens.agent_recovery`](/consul/docs/reference/agent#acl_tokens_agent_recovery) to allow +configured with [`acl.tokens.agent_recovery`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_agent_recovery) to allow write access to these operations even if no ACL resolution capability is available. ## Event Rules @@ -245,7 +245,7 @@ read-only access to any event, and firing of the "deploy" event. The [`consul exec`](/consul/commands/exec) command uses events with the "\_rexec" prefix during operation, so to enable this feature in a Consul environment with ACLs enabled, you will need to give agents a token with access to this event prefix, in addition to configuring -[`disable_remote_exec`](/consul/docs/reference/agent#disable_remote_exec) to `false`. +[`disable_remote_exec`](/consul/docs/reference/agent/configuration-file/general#disable_remote_exec) to `false`. ## Identity Rules @@ -444,7 +444,7 @@ Refer to [Admin Partition Rules](#admin-partition-rules) for another example ru The `namespace` and `namespace_prefix` resource controls access to Consul namespaces. Namespaces define a scope of resources for which ACL rules apply. ACL rules, themselves, can then be defined to only to apply to specific namespaces. - + The ability to add many types of resources to separate namespaces was added to [Consul Enterprise](https://www.hashicorp.com/consul) 1.7.0. @@ -618,16 +618,16 @@ node "admin" { Agents must be configured with `write` privileges for their own node name so that the agent can register their node metadata, tagged addresses, and other information in the catalog. If configured incorrectly, the agent will print an error to the console when it tries to sync its state with the catalog. -Configure `write` access in the [`acl.tokens.agent`](/consul/docs/reference/agent#acl_tokens_agent) parameter. +Configure `write` access in the [`acl.tokens.agent`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_agent) parameter. -The [`acl.token.default`](/consul/docs/reference/agent#acl_tokens_default) used by the agent should have `read` access to a given node so that the DNS interface can be queried. +The [`acl.token.default`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_default) used by the agent should have `read` access to a given node so that the DNS interface can be queried. Node rules are used to filter query results when reading from the catalog or retrieving information from the health endpoints. This allows for configurations where a token has access to a given service name, but only on an allowed subset of node names. Consul agents check tokens locally when health checks are registered and when Consul performs periodic [anti-entropy](/consul/docs/concept/consistency) syncs. These actions may required an ACL token to complete. Use the following methods to configure ACL tokens for registration events: -* Configure a global token in the [acl.tokens.default](/consul/docs/reference/agent#acl_tokens_default) parameter. +* Configure a global token in the [acl.tokens.default](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_default) parameter. This allows a single token to be used during all check registration operations. * Provide an ACL token with `service` and `check` definitions at registration time. This allows for greater flexibility and enables the use of multiple tokens on the same agent. @@ -841,7 +841,7 @@ service "admin" { Consul's DNS interface is affected by restrictions on service rules. If the -[`acl.tokens.default`](/consul/docs/reference/agent#acl_tokens_default) used by the agent does not have `read` access to a +[`acl.tokens.default`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_default) used by the agent does not have `read` access to a given service, then the DNS interface will return no records when queried for it. When reading from the catalog or retrieving information from the health endpoints, service rules are @@ -853,7 +853,7 @@ performs periodic [anti-entropy](/consul/docs/concept/consistency) syncs, which ACL token to complete. To accommodate this, Consul provides two methods of configuring ACL tokens to use for registration events: -1. Using the [acl.tokens.default](/consul/docs/reference/agent#acl_tokens_default) configuration +1. Using the [acl.tokens.default](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_default) configuration directive. This allows a single token to be configured globally and used during all service and check registration operations. 2. Providing an ACL token with service and check definitions at registration @@ -868,8 +868,8 @@ to use for registration events: access. In addition to ACLs, in Consul 0.9.0 and later, the agent must be configured with -[`enable_script_checks`](/consul/docs/reference/agent#enable_script_checks) or -[`enable_local_script_checks`](/consul/docs/reference/agent#enable_local_script_checks) +[`enable_script_checks`](/consul/docs/reference/agent/configuration-file/general#enable_script_checks) or +[`enable_local_script_checks`](/consul/docs/reference/agent/configuration-file/general#enable_local_script_checks) set to `true` in order to enable script checks. ### Reading Imported Services diff --git a/website/content/docs/reference/agent/telemetry.mdx b/website/content/docs/reference/agent/telemetry.mdx index 609bb6f33d4..14ec9dbdcc3 100644 --- a/website/content/docs/reference/agent/telemetry.mdx +++ b/website/content/docs/reference/agent/telemetry.mdx @@ -86,7 +86,7 @@ These metrics are used to monitor the health of the Consul servers. | `consul.raft.applied_index` | Represents the raft applied index. | index | gauge | | `consul.raft.apply` | Counts the number of Raft transactions occurring over the interval, which is a general indicator of the write load on the Consul servers. | raft transactions / interval | counter | | `consul.raft.barrier` | Counts the number of times the agent has started the barrier i.e the number of times it has issued a blocking call, to ensure that the agent has all the pending operations that were queued, to be applied to the agent's FSM. | blocks / interval | counter | -| `consul.raft.boltdb.freelistBytes` | Represents the number of bytes necessary to encode the freelist metadata. When [`raft_logstore.boltdb.no_freelist_sync`](/consul/docs/reference/agent#raft_logstore_boltdb_no_freelist_sync) is set to `false` these metadata bytes must also be written to disk for each committed log. | bytes | gauge | +| `consul.raft.boltdb.freelistBytes` | Represents the number of bytes necessary to encode the freelist metadata. When [`raft_logstore.boltdb.no_freelist_sync`](/consul/docs/reference/agent/configuration-file/raft#raft_logstore_boltdb_no_freelist_sync) is set to `false` these metadata bytes must also be written to disk for each committed log. | bytes | gauge | | `consul.raft.boltdb.freePageBytes` | Represents the number of bytes of free space within the raft.db file. | bytes | gauge | | `consul.raft.boltdb.getLog` | Measures the amount of time spent reading logs from the db. | ms | timer | | `consul.raft.boltdb.logBatchSize` | Measures the total size in bytes of logs being written to the db in a single batch. | bytes | sample | @@ -121,11 +121,11 @@ These metrics are used to monitor the health of the Consul servers. | `consul.raft.leader.dispatchLog` | Measures the time it takes for the leader to write log entries to disk. | ms | timer | | `consul.raft.leader.dispatchNumLogs` | Measures the number of logs committed to disk in a batch. | logs | gauge | | `consul.raft.logstore.verifier.checkpoints_written` | Counts the number of checkpoint entries written to the LogStore. | checkpoints | counter | -| `consul.raft.logstore.verifier.dropped_reports` | Counts how many times the verifier routine was still busy when the next checksum came in and so verification for a range was skipped. If you see this happen, consider increasing the interval between checkpoints with [`raft_logstore.verification.interval`](/consul/docs/reference/agent#raft_logstore_verification) | reports dropped | counter | +| `consul.raft.logstore.verifier.dropped_reports` | Counts how many times the verifier routine was still busy when the next checksum came in and so verification for a range was skipped. If you see this happen, consider increasing the interval between checkpoints with [`raft_logstore.verification.interval`](/consul/docs/reference/agent/configuration-file/raft#raft_logstore_verification) | reports dropped | counter | | `consul.raft.logstore.verifier.ranges_verified` | Counts the number of log ranges for which a verification report has been completed. Refer to [Monitor Raft metrics and logs for WAL](/consul/docs/deploy/server/wal/monitor-raft) for more information. | log ranges verifications | counter | | `consul.raft.logstore.verifier.read_checksum_failures` | Counts the number of times a range of logs between two check points contained at least one disk corruption. Refer to [Monitor Raft metrics and logs for WAL](/consul/docs/deploy/server/wal/monitor-raft) for more information. | disk corruptions | counter | | `consul.raft.logstore.verifier.write_checksum_failures` | Counts the number of times a follower has a different checksum to the leader at the point where it writes to the log. This could be caused by either a disk-corruption on the leader (unlikely) or some other corruption of the log entries in-flight. | in-flight corruptions | counter | -| `consul.raft.leader.lastContact` | Measures the time since the leader was last able to contact the follower nodes when checking its leader lease. It can be used as a measure for how stable the Raft timing is and how close the leader is to timing out its lease.The lease timeout is 500 ms times the [`raft_multiplier` configuration](/consul/docs/reference/agent#raft_multiplier), so this telemetry value should not be getting close to that configured value, otherwise the Raft timing is marginal and might need to be tuned, or more powerful servers might be needed. See the [Server Performance](/consul/docs/deploy/server/vm/requirements) guide for more details. | ms | timer | +| `consul.raft.leader.lastContact` | Measures the time since the leader was last able to contact the follower nodes when checking its leader lease. It can be used as a measure for how stable the Raft timing is and how close the leader is to timing out its lease.The lease timeout is 500 ms times the [`raft_multiplier` configuration](/consul/docs/reference/agent/configuration-file/raft#raft_multiplier), so this telemetry value should not be getting close to that configured value, otherwise the Raft timing is marginal and might need to be tuned, or more powerful servers might be needed. See the [Server Performance](/consul/docs/deploy/server/vm/requirements) guide for more details. | ms | timer | | `consul.raft.leader.oldestLogAge` | The number of milliseconds since the _oldest_ log in the leader's log store was written. This can be important for replication health where write rate is high and the snapshot is large as followers may be unable to recover from a restart if restoring takes longer than the minimum value for the current leader. Compare this with `consul.raft.fsm.lastRestoreDuration` and `consul.raft.rpc.installSnapshot` to monitor. In normal usage this gauge value will grow linearly over time until a snapshot completes on the leader and the log is truncated. Note: this metric won't be emitted until the leader writes a snapshot. After an upgrade to Consul 1.10.0 it won't be emitted until the oldest log was written after the upgrade. | ms | gauge | | `consul.raft.replication.heartbeat` | Measures the time taken to invoke appendEntries on a peer, so that it doesn't timeout on a periodic basis. | ms | timer | | `consul.raft.replication.appendEntries` | Measures the time it takes to replicate log entries to followers. This is a general indicator of the load pressure on the Consul servers, as well as the performance of the communication between the servers. | ms | timer | diff --git a/website/content/docs/reference/architecture/ports.mdx b/website/content/docs/reference/architecture/ports.mdx index a6092fa5c8a..e6b2e75b4b7 100644 --- a/website/content/docs/reference/architecture/ports.mdx +++ b/website/content/docs/reference/architecture/ports.mdx @@ -8,7 +8,7 @@ description: Find information about the ports that Consul requires for its netwo This page provides reference information about the required ports that Consul exposes for its operations. -You can change or disable Consul's default ports in the [agent configuration file](/consul/docs/reference/agent#ports) or using the [`consul agent` CLI command](/consul/commands/agent). +You can change or disable Consul's default ports in the [agent configuration file](/consul/docs/reference/agent/configuration-file/general#ports) or using the [`consul agent` CLI command](/consul/commands/agent). ## Overview @@ -77,7 +77,7 @@ This port receives incoming traffic from workloads that make HTTPS API calls. The server HTTPS port does not need to be open when Consul clients service all HTTPS API calls. Consul does not use this port for internal communication between servers, clients, dataplanes, gateways, and Envoy proxies. -This port is disabled by default. You can enable it in the [agent configuration file](/consul/docs/reference/agent#ports) or using the [`consul agent` CLI command](/consul/commands/agent). +This port is disabled by default. You can enable it in the [agent configuration file](/consul/docs/reference/agent/configuration-file/general#ports) or using the [`consul agent` CLI command](/consul/commands/agent). HCP Consul assigns port `443` to HashiCorp-managed clusters, instead of the default `8501`. @@ -91,7 +91,7 @@ The following table lists information about the Consul API's gRPC port defaults: When using [Consul Dataplane](/consul/docs/architecture/control-plane/dataplane), this port receives incoming traffic from the dataplanes. -We recommend using gRPC TLS instead, so this port is disabled by default. You can enable it in the [agent configuration file](/consul/docs/reference/agent#ports) or using the [`consul agent` CLI command](/consul/commands/agent). +We recommend using gRPC TLS instead, so this port is disabled by default. You can enable it in the [agent configuration file](/consul/docs/reference/agent/configuration-file/general#ports) or using the [`consul agent` CLI command](/consul/commands/agent). HCP Consul does not support the gRPC port. @@ -203,7 +203,7 @@ The following table lists information about the Consul client's HTTPS port defau This port receives incoming traffic from workloads that make HTTPS API calls. Consul does not use this port for internal communication between servers, clients, dataplanes, gateways, and Envoy proxies. -This port is disabled by default. You can enable it in the [agent configuration file](/consul/docs/reference/agent#ports) or using the [`consul agent` CLI command](/consul/commands/agent). +This port is disabled by default. You can enable it in the [agent configuration file](/consul/docs/reference/agent/configuration-file/general#ports) or using the [`consul agent` CLI command](/consul/commands/agent). When this port is enabled, the Consul CLI uses it to interact with Consul. @@ -229,7 +229,7 @@ The following table lists information about the Consul client's gRPC with TLS po This port receives incoming traffic from the gateways and Envoy proxies registered to this client. We recommend using `8503` as your conventional gRPC port number, as it allows some tools to work automatically. -This port is disabled by default. You can enable it in the [agent configuration file](/consul/docs/reference/agent#ports) or using the [`consul agent` CLI command](/consul/commands/agent). +This port is disabled by default. You can enable it in the [agent configuration file](/consul/docs/reference/agent/configuration-file/general#ports) or using the [`consul agent` CLI command](/consul/commands/agent). ### Client LAN serf diff --git a/website/content/docs/reference/config-entry/exported-services.mdx b/website/content/docs/reference/config-entry/exported-services.mdx index 01c56b3a47d..a7f3e255b34 100644 --- a/website/content/docs/reference/config-entry/exported-services.mdx +++ b/website/content/docs/reference/config-entry/exported-services.mdx @@ -22,7 +22,7 @@ You can configure the settings defined in the `exported-services` configuration ## Usage 1. Verify that your datacenter meets the conditions specified in the [Requirements](#requirements). -1. Specify the `exported-services` configuration in the agent configuration file (refer to [`config_entries`](/consul/docs/reference/agent#config_entries)) as described in [Configuration](#configuration). +1. Specify the `exported-services` configuration in the agent configuration file (refer to [`config_entries`](/consul/docs/reference/agent/configuration-file/general#config_entries)) as described in [Configuration](#configuration). 1. Apply the configuration using one of the following methods: - Kubernetes CRD: Refer to the [Custom Resource Definitions](/consul/docs/fundamentals/config-entry) documentation for details. - Issue the `consul config write` command: Refer to the [Consul Config Write](/consul/commands/config/write) documentation for details. diff --git a/website/content/docs/reference/cts/index.mdx b/website/content/docs/reference/cts/index.mdx index 6849061481b..132baefcd0d 100644 --- a/website/content/docs/reference/cts/index.mdx +++ b/website/content/docs/reference/cts/index.mdx @@ -87,7 +87,7 @@ license { You can use the `auto_retrieval` block to configure the automatic license retrieval in CTS. When enabled, CTS attempts to retrieve a new license from its configured Consul Enterprise backend once a day. If CTS cannot retrieve a license and the current license is reaching its expiration date, CTS attempts to retrieve a license with increased frequency, as defined by the [License Expiration Date Handling](/consul/docs/nia/enterprise/license#license-expiration-handling). - + Enabling `auto_retrieval` is recommended when using HCP Consul, as HCP Consul licenses expire more frequently than Consul Enterprise licenses. Without auto-retrieval enabled, you have to restart CTS every time you load a new license. @@ -100,9 +100,9 @@ Enabling `auto_retrieval` is recommended when using HCP Consul, as HCP Consul li The `consul` block configures the CTS connection with a Consul agent so that CTS can perform queries for task execution. It also configures the automatic registration of CTS as a service with Consul. - - -Use HTTP/2 to improve Consul-Terraform-Sync performance when communicating with the local Consul process. [TLS/HTTPS](/consul/docs/reference/agent) must be configured for the local Consul with the [cert_file](/consul/docs/reference/agent#cert_file) and [key_file](/consul/docs/reference/agent#key_file) parameters set. For the Consul-Terraform-Sync configuration, set `tls.enabled = true` and set the `address` parameter to the HTTPS URL, e.g., `address = example.consul.com:8501`. If using self-signed certificates for Consul, you will also need to set `tls.verify = false` or add the certificate to `ca_cert` or `ca_path`. + + +Use HTTP/2 to improve Consul-Terraform-Sync performance when communicating with the local Consul process. [TLS/HTTPS](/consul/docs/reference/agent) must be configured for the local Consul with the [cert_file](/consul/docs/reference/agent/configuration-file/tls#cert_file) and [key_file](/consul/docs/reference/agent/configuration-file/tls#key_file) parameters set. For the Consul-Terraform-Sync configuration, set `tls.enabled = true` and set the `address` parameter to the HTTPS URL, e.g., `address = example.consul.com:8501`. If using self-signed certificates for Consul, you will also need to set `tls.verify = false` or add the certificate to `ca_cert` or `ca_path`. @@ -161,7 +161,7 @@ Configures HTTP basic authentication for communicating with Consul. ### TLS -Configure TLS to use a secure client connection with Consul. Using HTTP/2 can solve issues related to hitting Consul's maximum connection limits, as well as improve efficiency when processing many blocking queries. This option is required for Consul-Terraform-Sync when connecting to a [Consul agent with TLS verification enabled for HTTPS connections](/consul/docs/reference/agent#verify_incoming). +Configure TLS to use a secure client connection with Consul. Using HTTP/2 can solve issues related to hitting Consul's maximum connection limits, as well as improve efficiency when processing many blocking queries. This option is required for Consul-Terraform-Sync when connecting to a [Consul agent with TLS verification enabled for HTTPS connections](/consul/docs/reference/agent/configuration-file/tls#verify_incoming). If Consul is using a self-signed certificate that you have not added to the global CA chain, you can set this certificate with `ca_cert` or `ca_path`. Alternatively, you can disable SSL verification by setting `verify` to false. However, disabling verification is a potential security vulnerability. @@ -179,7 +179,7 @@ If Consul is using a self-signed certificate that you have not added to the glob Configures the low-level network connection details to Consul. -To achieve the shortest latency between a Consul service update to a task execution, configure `max_idle_conns_per_host` equal to or greater than the number of services in automation across all tasks. This value should be lower than the configured [`http_max_conns_per_client`](/consul/docs/reference/agent#http_max_conns_per_client) for the Consul agent. +To achieve the shortest latency between a Consul service update to a task execution, configure `max_idle_conns_per_host` equal to or greater than the number of services in automation across all tasks. This value should be lower than the configured [`http_max_conns_per_client`](/consul/docs/reference/agent/configuration-file/general#http_max_conns_per_client) for the Consul agent. If `max_idle_conns_per_host` and the number of services in automation is greater than the Consul agent limit, CTS may error due to connection limits (status code 429). You may increase the agent limit with caution. _Requests to the Consul agent made by Terraform subprocesses or any other process on the same host as CTS will contribute to the Consul agent connection limit._ @@ -258,7 +258,7 @@ The `instance` parameter is an object that contains configurations unique to the ## Service - + Deprecated in CTS 0.5.0 and will be removed in a future major release. `service` blocks are used to define the `task` block's `services` fields, which were also deprecated and replaced with [Services Condition](/consul/docs/automate/infrastructure/configure#services-condition) and [Services Module Input](/consul/docs/automate/infrastructure/configure#services-module-input). `service` block configuration can be replaced by configuring the equivalent fields of the corresponding Services Condition and Services Module Input. Refer to [0.5.0 release notes](/consul/docs/release-notes/consul-terraform-sync/v0_5_x#deprecate-service-block) for examples. @@ -530,7 +530,7 @@ task { ### Task Module Input ((#task-source-input)) - + `module_input` was renamed from `source_input` in CTS 0.5.0. Documentation for the `module_input` block also applies to the `source_input` block. @@ -782,7 +782,7 @@ task { ``` - + Provider arguments configured in CTS configuration files are written in plain text to the generated [`terraform.tfvars`](/consul/docs/nia/network-drivers#terraform-tfvars) file for each Terraform workspace that references the provider. To exclude arguments or dynamic values from rendering to local files in plain text, use [`task_env` in addition to using dynamic configuration](#securely-configure-terraform-providers). @@ -820,7 +820,7 @@ terraform_provider "foo" { ``` - + CTS does not prevent sensitive values from being written to Terraform state files. We recommend securing state files in addition to securely configuring Terraform providers. Options for securing state files can be set within [`driver.backend`](#backend) based on the backend used. For example, Consul KV is the default backend and can be secured with ACLs for KV path. For other backends, we recommend enabling encryption, if applicable. @@ -882,7 +882,7 @@ terraform_provider "example" { - `unwrap_token` - (bool) Unwraps the provided Vault token as a wrapped token. - + Vault credentials are not accessible by tasks and the associated Terraform configurations, including automated Terraform modules. If the task requires Vault, you will need to separately configure the Vault provider and explicitly include it in the `task.providers` list. diff --git a/website/content/docs/reference/proxy/built-in.mdx b/website/content/docs/reference/proxy/built-in.mdx index 6eabfafd092..eda3871dda9 100644 --- a/website/content/docs/reference/proxy/built-in.mdx +++ b/website/content/docs/reference/proxy/built-in.mdx @@ -8,7 +8,7 @@ description: >- # Built-in proxy configuration reference - + The built-in proxy is not supported for production deployments. It does not support many of Consul's service mesh features, and is not under active development. The [Envoy proxy](/consul/docs/reference/proxy/envoy) should be used for production deployments. @@ -58,8 +58,8 @@ All fields are optional with a reasonable default. - `bind_port` - The port the proxy will bind its _public_ mTLS listener to. If not provided, the agent will assign a random port from its - configured proxy port range specified by [`sidecar_min_port`](/consul/docs/reference/agent#sidecar_min_port) - and [`sidecar_max_port`](/consul/docs/reference/agent#sidecar_max_port). + configured proxy port range specified by [`sidecar_min_port`](/consul/docs/reference/agent/configuration-file/general#sidecar_min_port) + and [`sidecar_max_port`](/consul/docs/reference/agent/configuration-file/general#sidecar_max_port). - `local_service_address`- The `[address]:port` that the proxy should use to connect to the local application instance. By default diff --git a/website/content/docs/reference/proxy/connect-proxy.mdx b/website/content/docs/reference/proxy/connect-proxy.mdx index f0cb6fe5fa8..c1de39fa4ad 100644 --- a/website/content/docs/reference/proxy/connect-proxy.mdx +++ b/website/content/docs/reference/proxy/connect-proxy.mdx @@ -178,7 +178,7 @@ You can configure the service mesh proxy to create listeners for upstream servic ### Upstream configuration examples -Upstreams support multiple destination types. The following examples include information about each implementation. Note that the examples in this topic use snake case, which is a convention that separates words with underscores, because the format is supported in configuration files and API registrations. +Upstreams support multiple destination types. The following examples include information about each implementation. Note that the examples in this topic use snake case, which is a convention that separates words with underscores, because the format is supported in configuration files and API registrations. @@ -315,7 +315,7 @@ Note that the examples in this topic use snake case, which is a convention that directly by transparent proxies. Transparent proxies typically dial upstreams using the "virtual" tagged address, which load balances across instances. A database cluster with a leader is an example where dialing individual instances can be helpful. Cannot be used with upstreams which define a `destination_peer`. - + Dynamic routing rules such as failovers and redirects do not apply to services dialed directly. Additionally, the connection is proxied using a TCP proxy with a connection timeout of 5 seconds. @@ -382,7 +382,7 @@ The following examples show possible configurations to expose HTTP paths through Exposing paths through Envoy enables a service to protect itself by only listening on localhost, while still allowing non-mesh-enabled applications to contact an HTTP endpoint. Some examples include: exposing a `/metrics` path for Prometheus or `/healthz` for kubelet liveness checks. - + Note that the examples in this topic use snake case, which is a convention that separates words with underscores, because the format is supported in configuration files and API registrations. #### Expose listeners in Envoy for health checks @@ -397,7 +397,7 @@ The following example exposes Envoy listeners to HTTP and GRPC checks registered } ``` -#### Expose an HTTP listener +#### Expose an HTTP listener The following example exposes and HTTP listener in Envoy at port `21500` that routes to an HTTP server listening at port `8080`: @@ -415,7 +415,7 @@ The following example exposes and HTTP listener in Envoy at port `21500` that ro } ``` -#### Expose an HTTP2 listener +#### Expose an HTTP2 listener The following example expose an HTTP2 listener in Envoy at port `21501` that routes to a gRPC server listening at port `9090`: @@ -436,8 +436,8 @@ The following example expose an HTTP2 listener in Envoy at port `21501` that rou - `checks` `(bool: false)` - If enabled, all HTTP and gRPC checks registered with the agent are exposed through Envoy. Envoy will expose listeners for these checks and will only accept connections originating from localhost or Consul's - [advertise address](/consul/docs/reference/agent#advertise). The port for these listeners are dynamically allocated from - [expose_min_port](/consul/docs/reference/agent#expose_min_port) to [expose_max_port](/consul/docs/reference/agent#expose_max_port). + [advertise address](/consul/docs/reference/agent/configuration-file/advertise). The port for these listeners are dynamically allocated from + [expose_min_port](/consul/docs/reference/agent/configuration-file/general#expose_min_port) to [expose_max_port](/consul/docs/reference/agent/configuration-file/general#expose_max_port). This flag is useful when a Consul client cannot reach registered services over localhost. One example is when running Consul on Kubernetes, and Consul agents run in their own pods. - `paths` `array: []` - A list of paths to expose through Envoy. diff --git a/website/content/docs/reference/proxy/sidecar.mdx b/website/content/docs/reference/proxy/sidecar.mdx index 58b2043061d..3806abb8497 100644 --- a/website/content/docs/reference/proxy/sidecar.mdx +++ b/website/content/docs/reference/proxy/sidecar.mdx @@ -7,7 +7,7 @@ description: >- # Sidecar proxy configuration reference -The `sidecar_service` block is a service definition that can contain most regular service definition fields. Refer to [Limitations](#limitations) for information about unsupported service definition fields for sidecar proxies. +The `sidecar_service` block is a service definition that can contain most regular service definition fields. Refer to [Limitations](#limitations) for information about unsupported service definition fields for sidecar proxies. Consul treats sidecar proxy service definitions as a root-level service definition. All fields are optional in nested definitions, which default to opinionated settings that are intended to reduce burden of setting up a sidecar proxy. @@ -26,8 +26,8 @@ proxy. - `tags` - Defaults to the tags of the parent service. - `meta` - Defaults to the service metadata of the parent service. - `port` - Defaults to being auto-assigned from a configurable - range specified by [`sidecar_min_port`](/consul/docs/reference/agent#sidecar_min_port) - and [`sidecar_max_port`](/consul/docs/reference/agent#sidecar_max_port). + range specified by [`sidecar_min_port`](/consul/docs/reference/agent/configuration-file/general#sidecar_min_port) + and [`sidecar_max_port`](/consul/docs/reference/agent/configuration-file/general#sidecar_max_port). - `kind` - Defaults to `connect-proxy`. This value cannot be overridden. - `check`, `checks` - By default we add a TCP check on the local address and port for the proxy, and a [service alias @@ -43,7 +43,7 @@ proxy. In the following example, the `sidecar_service` macro sets baselines configurations for the proxy, but the [proxy upstreams](/consul/docs/reference/proxy/connect-proxy#upstream-configuration-reference) and [built-in proxy -configuration](/consul/docs/reference/proxy/built-in) fields contain custom values: +configuration](/consul/docs/reference/proxy/built-in) fields contain custom values: ```json { diff --git a/website/content/docs/register/health-check/vm.mdx b/website/content/docs/register/health-check/vm.mdx index 006ec4ba3a2..fabe0c1c1b3 100644 --- a/website/content/docs/register/health-check/vm.mdx +++ b/website/content/docs/register/health-check/vm.mdx @@ -7,11 +7,11 @@ description: >- # Define health checks -This topic describes how to create different types of health checks for your services. +This topic describes how to create different types of health checks for your services. ## Overview -Health checks are configurations that verifies the health of a service or node. Health checks configurations are nested in the `service` block. Refer to [Define Services](/consul/docs/register/service/vm/define) for information about specifying other service parameters. +Health checks are configurations that verifies the health of a service or node. Health checks configurations are nested in the `service` block. Refer to [Define Services](/consul/docs/register/service/vm/define) for information about specifying other service parameters. You can define individual health checks for your service in separate `check` blocks or define multiple checks in a `checks` block. Refer to [Define multiple checks](#define-multiple-checks) for additional information. @@ -19,15 +19,15 @@ You can create several different kinds of checks: - _Script_ checks invoke an external application that performs the health check, exits with an appropriate exit code, and potentially generates output. Script checks are one of the most common types of checks. - _HTTP_ checks make an HTTP GET request to the specified URL and wait for the specified amount of time. HTTP checks are one of the most common types of checks. -- _TCP_ checks attempt to connect to an IP or hostname and port over TCP and wait for the specified amount of time. -- _UDP_ checks send UDP datagrams to the specified IP or hostname and port and wait for the specified amount of time. -- _Time-to-live (TTL)_ checks are passive checks that await updates from the service. If the check does not receive a status update before the specified duration, the health check enters a `critical`state. -- _Docker_ checks are dependent on external applications packaged with a Docker container that are triggered by calls to the Docker `exec` API endpoint. -- _gRPC_ checks probe applications that support the standard gRPC health checking protocol. -- _H2ping_ checks test an endpoint that uses http2. The check connects to the endpoint and sends a ping frame. -- _Alias_ checks represent the health state of another registered node or service. +- _TCP_ checks attempt to connect to an IP or hostname and port over TCP and wait for the specified amount of time. +- _UDP_ checks send UDP datagrams to the specified IP or hostname and port and wait for the specified amount of time. +- _Time-to-live (TTL)_ checks are passive checks that await updates from the service. If the check does not receive a status update before the specified duration, the health check enters a `critical`state. +- _Docker_ checks are dependent on external applications packaged with a Docker container that are triggered by calls to the Docker `exec` API endpoint. +- _gRPC_ checks probe applications that support the standard gRPC health checking protocol. +- _H2ping_ checks test an endpoint that uses http2. The check connects to the endpoint and sends a ping frame. +- _Alias_ checks represent the health state of another registered node or service. -If your network runs in a Kubernetes environment, you can sync service health information with Kubernetes health checks. Refer to [Configure Health Checks for Consul on Kubernetes](/consul/docs/register/health-check/k8s) for details. +If your network runs in a Kubernetes environment, you can sync service health information with Kubernetes health checks. Refer to [Configure Health Checks for Consul on Kubernetes](/consul/docs/register/health-check/k8s) for details. ### Registration @@ -127,7 +127,7 @@ check = { Script checks invoke an external application that performs the health check, exits with an appropriate exit code, and potentially generates output data. The output of a script check is limited to 4KB. Outputs that exceed the limit are truncated. -Script checks timeout after 30 seconds by default, but you can configure a custom script check timeout value by specifying the `timeout` field in the check definition. When the timeout is reached on Windows, Consul waits for any child processes spawned by the script to finish. For any other system, Consul attempts to force-kill the script and any child processes it has spawned once the timeout has passed. +Script checks timeout after 30 seconds by default, but you can configure a custom script check timeout value by specifying the `timeout` field in the check definition. When the timeout is reached on Windows, Consul waits for any child processes spawned by the script to finish. For any other system, Consul attempts to force-kill the script and any child processes it has spawned once the timeout has passed. ### Script check configuration @@ -184,9 +184,9 @@ The following exit codes returned by the script check determine the health check Any output of the script is captured and made available in the `Output` field of checks included in HTTP API responses. Refer to the example described in the [local service health endpoint](/consul/api-docs/agent/service#by-name-json). -## HTTP checks +## HTTP checks -_HTTP_ checks send an HTTP request to the specified URL and report the service health based on the [HTTP response code](#http-check-response-codes). We recommend using HTTP checks over [script checks](#script-checks) that use cURL or another external process to check an HTTP operation. +_HTTP_ checks send an HTTP request to the specified URL and report the service health based on the [HTTP response code](#http-check-response-codes). We recommend using HTTP checks over [script checks](#script-checks) that use cURL or another external process to check an HTTP operation. ### HTTP check configuration @@ -232,7 +232,7 @@ check = { ``` -HTTP checks send GET requests by default, but you can specify another request method in the `method` field. You can send additional headers in the `header` block. The `header` block contains a key and an array of strings, such as `{"x-foo": ["bar", "baz"]}`. By default, HTTP checks timeout at 10 seconds, but you can specify a custom timeout value in the `timeout` field. +HTTP checks send GET requests by default, but you can specify another request method in the `method` field. You can send additional headers in the `header` block. The `header` block contains a key and an array of strings, such as `{"x-foo": ["bar", "baz"]}`. By default, HTTP checks timeout at 10 seconds, but you can specify a custom timeout value in the `timeout` field. HTTP checks expect a valid TLS certificate by default. You can disable certificate verification by setting the `tls_skip_verify` field to `true`. When using TLS and a host name is specified in the `http` field, the check automatically determines the SNI from the URL. If the `http` field is configured with an IP address or if you want to explicitly set the SNI, specify the name in the `tls_server_name` field. @@ -240,15 +240,15 @@ The check follows HTTP redirects configured in the network by default. Set the ` ### HTTP check response codes -Responses larger than 4KB are truncated. The HTTP response determines the status of the service: +Responses larger than 4KB are truncated. The HTTP response determines the status of the service: - A `200`-`299` response code is healthy. -- A `429` response code indicating too many requests is a warning. +- A `429` response code indicating too many requests is a warning. - All other response codes indicate a failure. ## TCP checks -TCP checks establish connections to the specified IPs or hosts. If the check successfully establishes a connection, the service status is reported as `success`. If the IP or host does not accept the connection, the service status is reported as `critical`. We recommend TCP checks over [script checks](#script-checks) that use netcat or another external process to check a socket operation. +TCP checks establish connections to the specified IPs or hosts. If the check successfully establishes a connection, the service status is reported as `success`. If the IP or host does not accept the connection, the service status is reported as `critical`. We recommend TCP checks over [script checks](#script-checks) that use netcat or another external process to check a socket operation. ### TCP check configuration @@ -283,7 +283,7 @@ check = { -If a hostname resolves to an IPv4 and an IPv6 address, Consul attempts to connect to both addresses. The first successful connection attempt results in a successful check. +If a hostname resolves to an IPv4 and an IPv6 address, Consul attempts to connect to both addresses. The first successful connection attempt results in a successful check. By default, TCP check requests timeout at 10 seconds, but you can specify a custom timeout in the `timeout` field. @@ -293,9 +293,9 @@ UDP checks direct the Consul agent to send UDP datagrams to the specified IP or ### UDP check configuration -Add a `udp` field to the `check` block in your service definition file and specify the address, including port number, for sending datagrams. All other fields are optional. Refer to [Health Checks Configuration Reference](/consul/docs/reference/service/health-check) for information about all health check configurations. +Add a `udp` field to the `check` block in your service definition file and specify the address, including port number, for sending datagrams. All other fields are optional. Refer to [Health Checks Configuration Reference](/consul/docs/reference/service/health-check) for information about all health check configurations. -In the following example, a UDP check named `DNS UDP on port 53` sends datagrams to `localhost:53` every 10 seconds: +In the following example, a UDP check named `DNS UDP on port 53` sends datagrams to `localhost:53` every 10 seconds: @@ -331,7 +331,7 @@ OSService checks if an OS service is running on the host. OSService checks suppo ### OSService check configurations -Add an `os_service` field to the `check` block in your service definition file and specify the name of the service to check. All other fields are optional. Refer to [Health Checks Configuration Reference](/consul/docs/reference/service/health-check) for information about all health check configurations. +Add an `os_service` field to the `check` block in your service definition file and specify the name of the service to check. All other fields are optional. Refer to [Health Checks Configuration Reference](/consul/docs/reference/service/health-check) for information about all health check configurations. In the following example, an OSService check named `svcname-001 Windows Service Health` verifies that the `myco-svctype-svcname-001` service is running every 10 seconds: @@ -365,18 +365,18 @@ check = { Time-to-live (TTL) checks wait for an external process to report the service's state to a Consul [`/agent/check` HTTP endpoint](/consul/api-docs/agent/check). If the check does not receive an update before the specified `ttl` duration, the check logs the service as `critical`. For example, if a healthy application is configured to periodically send a `PUT` request a status update to the HTTP endpoint, then the health check logs a `critical` state if the application is unable to send the update before the TTL expires. The check uses the following endpoints to update health information: -- [pass](/consul/api-docs/agent/check#ttl-check-pass) +- [pass](/consul/api-docs/agent/check#ttl-check-pass) - [warn] (/consul/api-docs/agent/check#ttl-check-warn) -- [fail](/consul/api-docs/agent/check#ttl-check-fail) -- [update](/consul/api-docs/agent/check#ttl-check-update) +- [fail](/consul/api-docs/agent/check#ttl-check-fail) +- [update](/consul/api-docs/agent/check#ttl-check-update) -TTL checks also persist their last known status to disk so that the Consul agent can restore the last known status of the check across restarts. Persisted check status is valid through the end of the TTL from the time of the last check. +TTL checks also persist their last known status to disk so that the Consul agent can restore the last known status of the check across restarts. Persisted check status is valid through the end of the TTL from the time of the last check. You can manually mark a service as unhealthy using the [`consul maint` CLI command](/consul/commands/maint) or [`agent/maintenance` HTTP API endpoint](/consul/api-docs/agent#enable-maintenance-mode), rather than waiting for a TTL health check if the `ttl` duration is high. ### TTL check configuration -Add a `ttl` field to the `check` block in your service definition file and specify how long to wait for an update from the external process. All other fields are optional. Refer to [Health Checks Configuration Reference](/consul/docs/reference/service/health-check) for information about all health check configurations. +Add a `ttl` field to the `check` block in your service definition file and specify how long to wait for an update from the external process. All other fields are optional. Refer to [Health Checks Configuration Reference](/consul/docs/reference/service/health-check) for information about all health check configurations. In the following example, a TTL check named `Web App Status` logs the application as `critical` if a status update is not received every 30 seconds: @@ -407,9 +407,9 @@ check = { ## Docker checks -Docker checks invoke an application packaged within a Docker container. The application should perform a health check and exit with an appropriate exit code. +Docker checks invoke an application packaged within a Docker container. The application should perform a health check and exit with an appropriate exit code. -The application is triggered within the running container through the Docker `exec` API. You should have access to either the Docker HTTP API or the Unix socket. Consul uses the `$DOCKER_HOST` environment variable to determine the Docker API endpoint. +The application is triggered within the running container through the Docker `exec` API. You should have access to either the Docker HTTP API or the Unix socket. Consul uses the `$DOCKER_HOST` environment variable to determine the Docker API endpoint. The output of a Docker check is limited to 4KB. Larger outputs are truncated. @@ -420,7 +420,7 @@ To enable Docker checks, you must first enable the agent to send external reques 1. Add one of the following configurations to your agent configuration file to enable a Docker check: - [`enable_local_script_checks`](/consul/commands/agent#enable_local_script_checks): Enable script checks defined in local config files. Script checks registered using the HTTP API are not allowed. - + - [`enable_script_checks`](/consul/commands/agent#enable_script_checks): Enable script checks no matter how they are registered. !> **Security warning**: Enabling non-local script checks in some configurations may introduce a known remote execution vulnerability targeted by malware. We strongly recommend `enable_local_script_checks` instead. @@ -461,11 +461,11 @@ check = { -## gRPC checks -gRPC checks send a request to the specified endpoint. These checks are intended for applications that support the standard [gRPC health checking protocol](https://github.com/grpc/grpc/blob/master/doc/health-checking.md). +## gRPC checks +gRPC checks send a request to the specified endpoint. These checks are intended for applications that support the standard [gRPC health checking protocol](https://github.com/grpc/grpc/blob/master/doc/health-checking.md). ### gRPC check configuration -Add a `grpc` field to the `check` block in your service definition file and specify the endpoint, including port number, for sending requests. All other fields are optional. Refer to [Health Checks Configuration Reference](/consul/docs/reference/service/health-check) for information about all health check configurations. +Add a `grpc` field to the `check` block in your service definition file and specify the endpoint, including port number, for sending requests. All other fields are optional. Refer to [Health Checks Configuration Reference](/consul/docs/reference/service/health-check) for information about all health check configurations. In the following example, a gRPC check named `Service health status` probes the entire application by sending requests to `127.0.0.1:12345` every 10 seconds: @@ -526,9 +526,9 @@ check = { -TLS is disabled for gRPC checks by default. You can enable TLS by setting `grpc_use_tls` to `true`. If TLS is enabled, you must either provide a valid TLS certificate or disable certificate verification by setting the `tls_skip_verify` field to `true`. +TLS is disabled for gRPC checks by default. You can enable TLS by setting `grpc_use_tls` to `true`. If TLS is enabled, you must either provide a valid TLS certificate or disable certificate verification by setting the `tls_skip_verify` field to `true`. -By default, gRPC checks timeout after 10 seconds, but you can specify a custom duration in the `timeout` field. +By default, gRPC checks timeout after 10 seconds, but you can specify a custom duration in the `timeout` field. ## H2ping checks H2ping checks test an endpoint that uses HTTP2 by connecting to the endpoint and sending a ping frame. If the endpoint sends a response within the specified interval, the check status is set to `success`. @@ -565,18 +565,18 @@ check = { -TLS is enabled by default, but you can disable TLS by setting `h2ping_use_tls` to `false`. When TLS is disabled, the Consul sends pings over h2c. When TLS is enabled, a valid certificate is required unless `tls_skip_verify` is set to `true`. +TLS is enabled by default, but you can disable TLS by setting `h2ping_use_tls` to `false`. When TLS is disabled, the Consul sends pings over h2c. When TLS is enabled, a valid certificate is required unless `tls_skip_verify` is set to `true`. -By default, H2ping checks timeout at 10 seconds, but you can specify a custom duration in the `timeout` field. +By default, H2ping checks timeout at 10 seconds, but you can specify a custom duration in the `timeout` field. ## Alias checks -Alias checks continuously report the health state of another registered node or service. If the alias experiences errors while watching the actual node or service, the check reports a`critical` state. Consul updates the alias and actual node or service state asynchronously but nearly instantaneously. +Alias checks continuously report the health state of another registered node or service. If the alias experiences errors while watching the actual node or service, the check reports a`critical` state. Consul updates the alias and actual node or service state asynchronously but nearly instantaneously. -For aliased services on the same agent, the check monitors the local state without consuming additional network resources. For services and nodes on different agents, the check maintains a blocking query over the agent's connection with a current server and allows stale requests. +For aliased services on the same agent, the check monitors the local state without consuming additional network resources. For services and nodes on different agents, the check maintains a blocking query over the agent's connection with a current server and allows stale requests. ### ACLs -For the blocking query, the alias check presents the ACL token set on the actual service or the token configured in the check definition. If neither are available, the alias check falls back to the default ACL token set for the agent. Refer to [`acl.tokens.default`](/consul/docs/reference/agent#acl_tokens_default) for additional information about the default ACL token. +For the blocking query, the alias check presents the ACL token set on the actual service or the token configured in the check definition. If neither are available, the alias check falls back to the default ACL token set for the agent. Refer to [`acl.tokens.default`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_default) for additional information about the default ACL token. ### Alias checks configuration Add an `alias_service` field to the `check` block in your service definition file and specify the name of the service or node to alias. All other fields are optional. Refer to [Health Checks Configuration Reference](/consul/docs/reference/service/health-check) for information about all health check configurations. diff --git a/website/content/docs/secure-mesh/certificate/bootstrap.mdx b/website/content/docs/secure-mesh/certificate/bootstrap.mdx index 696f8c3b421..0d60272881a 100644 --- a/website/content/docs/secure-mesh/certificate/bootstrap.mdx +++ b/website/content/docs/secure-mesh/certificate/bootstrap.mdx @@ -8,7 +8,7 @@ description: >- # Bootstrap certificate authority on virtual machines (VM) Consul service mesh comes with a built-in Certificate Authority (CA) that will -bootstrap by default when you first [enable](/consul/docs/reference/agent#connect_enabled) Consul service mesh on your servers. +bootstrap by default when you first [enable](/consul/docs/reference/agent/configuration-file/service-mesh#connect_enabled) Consul service mesh on your servers. To use the built-in CA, enable it in the server's configuration. diff --git a/website/content/docs/secure-mesh/intention/create.mdx b/website/content/docs/secure-mesh/intention/create.mdx index b7b1949de71..1794785bbd1 100644 --- a/website/content/docs/secure-mesh/intention/create.mdx +++ b/website/content/docs/secure-mesh/intention/create.mdx @@ -24,7 +24,7 @@ Consul grants permissions for creating and managing intentions based on the dest Consul implicitly grants `intentions:read` permissions to destination services when they are configured with `service:read` or `service:write` permissions. This is so that the services can allow or deny inbound connections when they attempt to join the service mesh. Refer to [Service rules](/consul/docs/secure/acl/rule#service-rules) for additional information about configuring ACLs for intentions. -The default ACL policy configuration determines the default behavior for intentions. If the policy is set to `deny`, then all connections or requests are denied and you must enable them explicitly. Refer to [`default_policy`](/consul/docs/reference/agent#acl_default_policy) for details. +The default ACL policy configuration determines the default behavior for intentions. If the policy is set to `deny`, then all connections or requests are denied and you must enable them explicitly. Refer to [`default_policy`](/consul/docs/reference/agent/configuration-file/acl#acl_default_policy) for details. ## Create an intention @@ -175,4 +175,4 @@ You can apply the configuration entry using the [`consul config` command](/consu Refer to the following topics for details about applying configuration entries: - [How to Use Configuration Entries](/consul/docs/fundamentals/config-entry) -- [Custom Resource Definitions for Consul on Kubernetes](/consul/docs/fundamentals/config-entry) \ No newline at end of file +- [Custom Resource Definitions for Consul on Kubernetes](/consul/docs/fundamentals/config-entry) diff --git a/website/content/docs/secure-mesh/intention/index.mdx b/website/content/docs/secure-mesh/intention/index.mdx index c5b364a1a4a..cb756e73e40 100644 --- a/website/content/docs/secure-mesh/intention/index.mdx +++ b/website/content/docs/secure-mesh/intention/index.mdx @@ -2,7 +2,7 @@ layout: docs page_title: Service mesh intentions overview description: >- - Intentions are access controls that allow or deny incoming requests to services in the mesh. + Intentions are access controls that allow or deny incoming requests to services in the mesh. --- # Service intentions overview @@ -13,7 +13,7 @@ This topic provides overview information about Consul intentions, which are mech ## Intention types -Intentions control traffic communication between services at the network layer, also called _L4_ traffic, or the application layer, also called _L7 traffic_. The protocol that the destination service uses to send and receive traffic determines the type of authorization the intention can enforce. +Intentions control traffic communication between services at the network layer, also called _L4_ traffic, or the application layer, also called _L7 traffic_. The protocol that the destination service uses to send and receive traffic determines the type of authorization the intention can enforce. ### L4 traffic intentions @@ -27,15 +27,15 @@ If the destination service uses an HTTP-based protocol, then intentions can enfo ## Workflow -You can manually create intentions from the Consul UI, API, or CLI. You can also enable Consul to dynamically create them by defining traffic routes in service intention configuration entries. Refer to [Create and manage intentions](/consul/docs/secure-mesh/intention/create) for details. +You can manually create intentions from the Consul UI, API, or CLI. You can also enable Consul to dynamically create them by defining traffic routes in service intention configuration entries. Refer to [Create and manage intentions](/consul/docs/secure-mesh/intention/create) for details. ### Enforcement -The [proxy](/consul/docs/connect/proxy) or [natively-integrated application](/consul/docs/automate/native) enforces intentions on inbound connections or requests. Only one intention can control authorization between a pair of services at any single point in time. +The [proxy](/consul/docs/connect/proxy) or [natively-integrated application](/consul/docs/automate/native) enforces intentions on inbound connections or requests. Only one intention can control authorization between a pair of services at any single point in time. L4 intentions mediate the ability to establish new connections. Modifying an intention does not have an effect on existing connections. As a result, changing a connection from `allow` to `deny` does not sever the connection. -L7 intentions mediate the ability to issue new requests. When an intention is modified, requests received after the modification use the latest intention rules to enforce access. Changing a connection from `allow` to `deny` does not sever the connection, but doing so blocks new requests from being processed. +L7 intentions mediate the ability to issue new requests. When an intention is modified, requests received after the modification use the latest intention rules to enforce access. Changing a connection from `allow` to `deny` does not sever the connection, but doing so blocks new requests from being processed. ### Caching @@ -45,7 +45,7 @@ The intentions for services registered with a Consul agent are cached locally on Consul propagates updates to intentions almost instantly as a result of the continuous blocking query the agent uses. A _blocking query_ is a Consul API feature that uses long polling to wait for potential changes. Refer to [Blocking Queries](/consul/api-docs/features/blocking) for additional information. Proxies also use blocking queries to quickly update their local configurations. -Because all intention data is cached locally, authorizations for inbound connection persist, even if the agents are completely severed from the Consul servers or if the proxies are completely severed from their local Consul agent. If the connection is severed, Consul automatically applies changes to intentions when connectivity is restored. +Because all intention data is cached locally, authorizations for inbound connection persist, even if the agents are completely severed from the Consul servers or if the proxies are completely severed from their local Consul agent. If the connection is severed, Consul automatically applies changes to intentions when connectivity is restored. ### Intention maintenance @@ -53,12 +53,12 @@ Services should periodically call the [intention match API](/consul/api-docs/con ## Precedence and match order -Consul processes criteria defined in the service intention configuration entry to match incoming requests. When Consul finds a match, it applies the corresponding action specified in the configuration entry. The match criteria may include specific HTTP headers, request methods, or other attributes. Additionally, you can use regular expressions to programmatically match attributes. Refer to [Service intention configuration entry reference](/consul/docs/reference/config-entry/service-intentions) for details. +Consul processes criteria defined in the service intention configuration entry to match incoming requests. When Consul finds a match, it applies the corresponding action specified in the configuration entry. The match criteria may include specific HTTP headers, request methods, or other attributes. Additionally, you can use regular expressions to programmatically match attributes. Refer to [Service intention configuration entry reference](/consul/docs/reference/config-entry/service-intentions) for details. Consul orders the matches based the following factors: - Specificity: Incoming requests that match attributes directly have the highest precedence. For example, intentions that are configured to deny traffic from services that send `POST` requests take precedence over intentions that allow traffic from methods configured with the wildcard value `*`. -- Authorization: Consul enforces `deny` over `allow` if match criteria are weighted equally. +- Authorization: Consul enforces `deny` over `allow` if match criteria are weighted equally. The following table shows match precedence in descending order: @@ -74,8 +74,8 @@ The following table shows match precedence in descending order: | 2 | Exact | `*` | `*` | `*` | | 1 | `*` | `*` | `*` | `*` | -Consul prints the precedence value to the service intentions configuration entry after it processes the matching criteria. The value is read-only. Refer to -[`Precedence`](/consul/docs/reference/config-entry/service-intentions#precedence) for additional information. +Consul prints the precedence value to the service intentions configuration entry after it processes the matching criteria. The value is read-only. Refer to +[`Precedence`](/consul/docs/reference/config-entry/service-intentions#precedence) for additional information. Namespaces are an Enterprise feature. In Consul CE, the only allowable value for either namespace field is `"default"`. Other rows in the table are not applicable. @@ -85,6 +85,6 @@ target destination. After verifying the TLS client certificate, the cached intentions should be consulted for each incoming connection/request to determine if it should be accepted or rejected. -The default intention behavior is defined by the [`default_policy`](/consul/docs/reference/agent#acl_default_policy) configuration. +The default intention behavior is defined by the [`default_policy`](/consul/docs/reference/agent/configuration-file/acl#acl_default_policy) configuration. If the configuration is set `allow`, then all service-to-service connections in the mesh will be allowed by default. If is set to `deny`, then all connections or requests will be denied by default. diff --git a/website/content/docs/secure/acl/bootstrap.mdx b/website/content/docs/secure/acl/bootstrap.mdx index c7cf1e6e10e..68787f1dedb 100644 --- a/website/content/docs/secure/acl/bootstrap.mdx +++ b/website/content/docs/secure/acl/bootstrap.mdx @@ -24,7 +24,7 @@ For best results, we recommend bootstrapping your ACL system after deploying you ## Enable ACLs on the agents -Add [parameters for the ACL system](/consul/docs/reference/agent#acl) to the agent's configuration file and then restart the Consul service. You must apply the same parameters to every server and client in your Consul datacenter for Consul to correctly enable the ACL configuration. You should update the configuration files on all of your servers first, and then initiate a rolling restart. +Add [parameters for the ACL system](/consul/docs/reference/agent/configuration-file/acl) to the agent's configuration file and then restart the Consul service. You must apply the same parameters to every server and client in your Consul datacenter for Consul to correctly enable the ACL configuration. You should update the configuration files on all of your servers first, and then initiate a rolling restart. The following example enables Consul's ACLs and sets the default policy to `deny`. This setting means that only explicitly named resources are allowed to access Consul. The configuration also enables token persistence, which persists the token to disk and reloads it when an agent restarts. @@ -40,16 +40,16 @@ acl = { -The following table describes common agent configuration fields where ACLs are applicable and whether the configurations apply to servers, clients, or both. Refer to the [Agent reference docs](/consul/docs/reference/agent#acl) for a full list of configurable fields. +The following table describes common agent configuration fields where ACLs are applicable and whether the configurations apply to servers, clients, or both. Refer to the [Agent reference docs](/consul/docs/reference/agent/configuration-file/acl) for a full list of configurable fields. | Configuration option | Servers | Clients | Description | | -------------------- | ---------- | ---------- | ----------- | -| [`acl.enabled`](/consul/docs/reference/agent#acl_enabled) | `required` | `required` | Controls whether ACLs are enabled | -| [`acl.default_policy`](/consul/docs/reference/agent#acl_default_policy) | `optional` | `N/A` | Determines `allowlist` or `denylist` mode | -| [`acl.down_policy`](/consul/docs/reference/agent#acl_down_policy) | `optional` | `optional` | Determines what to do when the remote token or policy resolution fails | -| [`acl.role_ttl`](/consul/docs/reference/agent#acl_role_ttl) | `optional` | `optional` | Determines time-to-live for cached ACL roles | -| [`acl.policy_ttl`](/consul/docs/reference/agent#acl_policy_ttl) | `optional` | `optional` | Determines time-to-live for cached ACL policies | -| [`acl.token_ttl`](/consul/docs/reference/agent#acl_token_ttl) | `optional` | `optional` | Determines time-to-live for cached ACL tokens | +| [`acl.enabled`](/consul/docs/reference/agent/configuration-file/acl#acl_enabled) | `required` | `required` | Controls whether ACLs are enabled | +| [`acl.default_policy`](/consul/docs/reference/agent/configuration-file/acl#acl_default_policy) | `optional` | `N/A` | Determines `allowlist` or `denylist` mode | +| [`acl.down_policy`](/consul/docs/reference/agent/configuration-file/acl#acl_down_policy) | `optional` | `optional` | Determines what to do when the remote token or policy resolution fails | +| [`acl.role_ttl`](/consul/docs/reference/agent/configuration-file/acl#acl_role_ttl) | `optional` | `optional` | Determines time-to-live for cached ACL roles | +| [`acl.policy_ttl`](/consul/docs/reference/agent/configuration-file/acl#acl_policy_ttl) | `optional` | `optional` | Determines time-to-live for cached ACL policies | +| [`acl.token_ttl`](/consul/docs/reference/agent/configuration-file/acl#acl_token_ttl) | `optional` | `optional` | Determines time-to-live for cached ACL tokens | If you want to reduce Consul client restarts, you can enable the ACLs on them when [you apply the token to agents](#apply-individual-tokens-to-agents). @@ -101,18 +101,18 @@ Repeat this process for each agent. It is the ACL administrator's responsibility Apply the agent token to your Consul servers and ensure they are working correctly before applying the agent token to your Consul clients. -You can add the token to the Consul agent with any of the following methods: +You can add the token to the Consul agent with any of the following methods: - Add the token to the agent configuration file - Use the Consul CLI -- Use the Consul HTTP API +- Use the Consul HTTP API We recommend using the agent configuration file. That way if your agent restarts, it reloads the token from the agent configuration file. -To set the token in the [agent configuration file](/consul/docs/reference/agent#acl_tokens_agent), add the following snippet to your existing Consul configuration or create a new file in the Consul configuration directory. +To set the token in the [agent configuration file](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_agent), add the following snippet to your existing Consul configuration or create a new file in the Consul configuration directory. ```hcl acl { @@ -183,7 +183,7 @@ $ curl \ -You must to complete this process for every agent. +You must to complete this process for every agent. Once complete, every agent should have a token that has read and write permissions to Consul, but only for node-related actions. Actions for individual services are not yet allowed. @@ -193,7 +193,7 @@ If bootstrapping ACLs on an existing datacenter, remember to update the default -You must always add agent tokens to server agents using this process. However, Consul can automatically apply the agent tokens required by client agents with its auto-config feature. Auto-config involves issuing and validating secure JWTs using an OIDC identity provider (IdP). +You must always add agent tokens to server agents using this process. However, Consul can automatically apply the agent tokens required by client agents with its auto-config feature. Auto-config involves issuing and validating secure JWTs using an OIDC identity provider (IdP). ## Apply individual tokens to agents Add tokens to services @@ -339,7 +339,7 @@ service { -Refer to [Register services and health checks](/consul/docs/register) for instructions on registering services. If the service is already registered, re-register it with the token. +Refer to [Register services and health checks](/consul/docs/register) for instructions on registering services. If the service is already registered, re-register it with the token. ## Next steps @@ -355,4 +355,4 @@ Now that you have bootstrapped the ACL system and created tokens for agents and - [Create replication token](/consul/docs/secure/acl/token/replication) - [Create snapshot agent token](/consul/docs/secure/acl/token/snapshot-agent) - [Create Vault Consul storage backend token](/consul/docs/secure/acl/token/vault-backend) -- [Create Consul ESM token](/consul/docs/secure/acl/token/esm) \ No newline at end of file +- [Create Consul ESM token](/consul/docs/secure/acl/token/esm) diff --git a/website/content/docs/secure/acl/index.mdx b/website/content/docs/secure/acl/index.mdx index 17dd1db15f9..d81ec47bbdb 100644 --- a/website/content/docs/secure/acl/index.mdx +++ b/website/content/docs/secure/acl/index.mdx @@ -83,7 +83,7 @@ Refer to the following topics for details about service identities: ## Node identities -Node identities are configuration blocks that you can add to role configurations or specify when linking tokens to policies. You can use them during the authorization process to automatically generate a policy for specific nodes. When you configure the agent, you can specify the token linked to the policy with [`acl_tokens_agent`](/consul/docs/reference/agent#acl_tokens_agent). +Node identities are configuration blocks that you can add to role configurations or specify when linking tokens to policies. You can use them during the authorization process to automatically generate a policy for specific nodes. When you configure the agent, you can specify the token linked to the policy with [`acl_tokens_agent`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_agent). Node identities let you quickly construct policies for multiple nodes, rather than creating identical polices for each individual node. diff --git a/website/content/docs/secure/acl/policy.mdx b/website/content/docs/secure/acl/policy.mdx index 12ddaaa07ea..2ab018eccd0 100644 --- a/website/content/docs/secure/acl/policy.mdx +++ b/website/content/docs/secure/acl/policy.mdx @@ -154,7 +154,7 @@ Use the `policy` keyword and one of the following access levels to set a policy - `write`: Allows the resource to be read and modified. - `deny`: Denies read and write access to the resource. -The special `list` access level provides access to all keys with the specified resource label in the [Consul KV](/consul/commands/kv/). The `list` access level can only be used with the `key_prefix` resource. The [`acl.enable_key_list_policy`](/consul/docs/reference/agent#acl_enable_key_list_policy) setting must be set to `true`. +The special `list` access level provides access to all keys with the specified resource label in the [Consul KV](/consul/commands/kv/). The `list` access level can only be used with the `key_prefix` resource. The [`acl.enable_key_list_policy`](/consul/docs/reference/agent/configuration-file/acl#acl_enable_key_list_policy) setting must be set to `true`. ### Matching and prefix values @@ -242,7 +242,7 @@ Exact matching rules will only apply to the exact resource specified. The order ### Rule scope -A token's effective rule set includes the rules from all linked policies, including from roles and service identities. You can define policy rules in either an `allowlist` or `denylist` mode, depending on the configuration of the [`acl_default_policy`](/consul/docs/reference/agent#acl_default_policy). +A token's effective rule set includes the rules from all linked policies, including from roles and service identities. You can define policy rules in either an `allowlist` or `denylist` mode, depending on the configuration of the [`acl_default_policy`](/consul/docs/reference/agent/configuration-file/acl#acl_default_policy). If you have configured the default policy to deny access to all resources (recommended), you can specify `allowlist` in policy rules to explicitly allow access to resources. Conversely, if you have configured the default policy to allow access to all resources, you can specify `denylist` in policy rules to explicitly deny access to resources. diff --git a/website/content/docs/secure/acl/role.mdx b/website/content/docs/secure/acl/role.mdx index 98837c454b3..3691fbed685 100644 --- a/website/content/docs/secure/acl/role.mdx +++ b/website/content/docs/secure/acl/role.mdx @@ -72,7 +72,7 @@ Roles may contain the following attributes: You can specify a templated policy when configuring roles or linking tokens to policies. Templated policies enable you to quickly construct policies for common Consul use cases, rather than creating identical policies for each use cases. -Consul uses templated policies during the authorization process to automatically generate a policy for the use case specified. Consul links the generated policy to the role or token so that it will have permission for the specific use case. +Consul uses templated policies during the authorization process to automatically generate a policy for the use case specified. Consul links the generated policy to the role or token so that it will have permission for the specific use case. ### Templated policy specification @@ -98,7 +98,7 @@ The following templated policy example configuration uses the `builtin/service` Refer to the [API documentation for roles](/consul/api-docs/acl/roles#sample-payload) for additional information and examples. - + In Consul Enterprise, templated policies inherit the namespace or admin partition scope of the corresponding ACL token or role. @@ -199,7 +199,7 @@ Use the following syntax to define a service identity: Refer to the [API documentation for roles](/consul/api-docs/acl/roles#sample-payload) for additional information and examples. - + In Consul Enterprise, service identities inherit the namespace or admin partition scope of the corresponding ACL token or role. @@ -316,7 +316,7 @@ node_prefix "" { You can specify a node identity when configuring roles or linking tokens to policies. _Node_ commonly refers to a Consul agent, but a node can also be a physical server, cloud instance, virtual machine, or container. -Node identities enable you to quickly construct policies for nodes, rather than manually creating identical polices for each node. They are used during the authorization process to automatically generate a policy for the node(s) specified. You can specify the token linked to the policy in the [`acl_tokens_agent`](/consul/docs/reference/agent#acl_tokens_agent) field when configuring the agent. +Node identities enable you to quickly construct policies for nodes, rather than manually creating identical polices for each node. They are used during the authorization process to automatically generate a policy for the node(s) specified. You can specify the token linked to the policy in the [`acl_tokens_agent`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_agent) field when configuring the agent. ### Node identity specification @@ -340,7 +340,7 @@ Use the following syntax to define a node identity: Refer to the [API documentation for roles](/consul/api-docs/acl/roles#sample-payload) for additional information and examples. - + You can only apply node identities to tokens and roles in the `default` namespace. The generated policy rules allow for `service:read` permissions on all services in all namespaces. @@ -410,4 +410,4 @@ service_prefix "" { } ``` - \ No newline at end of file + diff --git a/website/content/docs/secure/acl/rule.mdx b/website/content/docs/secure/acl/rule.mdx index fe69c048a93..3548c345bc5 100644 --- a/website/content/docs/secure/acl/rule.mdx +++ b/website/content/docs/secure/acl/rule.mdx @@ -171,7 +171,7 @@ name that starts with `bar`. Since [Agent API](/consul/api-docs/agent) utility operations may be required before an agent is joined to a cluster, or during an outage of the Consul servers or ACL datacenter, a special token may be -configured with [`acl.tokens.agent_recovery`](/consul/docs/reference/agent#acl_tokens_agent_recovery) to allow +configured with [`acl.tokens.agent_recovery`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_agent_recovery) to allow write access to these operations even if no ACL resolution capability is available. ## Event Rules @@ -213,7 +213,7 @@ read-only access to any event, and firing of the "deploy" event. The [`consul exec`](/consul/commands/exec) command uses events with the "\_rexec" prefix during operation, so to enable this feature in a Consul environment with ACLs enabled, you will need to give agents a token with access to this event prefix, in addition to configuring -[`disable_remote_exec`](/consul/docs/reference/agent#disable_remote_exec) to `false`. +[`disable_remote_exec`](/consul/docs/reference/agent/configuration-file/general#disable_remote_exec) to `false`. ## Identity Rules @@ -412,7 +412,7 @@ Refer to [Admin Partition Rules](#admin-partition-rules) for another example ru The `namespace` and `namespace_prefix` resource controls access to Consul namespaces. Namespaces define a scope of resources for which ACL rules apply. ACL rules, themselves, can then be defined to only to apply to specific namespaces. - + The ability to add many types of resources to separate namespaces was added to [Consul Enterprise](https://www.hashicorp.com/consul) 1.7.0. @@ -586,16 +586,16 @@ node "admin" { Agents must be configured with `write` privileges for their own node name so that the agent can register their node metadata, tagged addresses, and other information in the catalog. If configured incorrectly, the agent will print an error to the console when it tries to sync its state with the catalog. -Configure `write` access in the [`acl.tokens.agent`](/consul/docs/reference/agent#acl_tokens_agent) parameter. +Configure `write` access in the [`acl.tokens.agent`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_agent) parameter. -The [`acl.token.default`](/consul/docs/reference/agent#acl_tokens_default) used by the agent should have `read` access to a given node so that the DNS interface can be queried. +The [`acl.token.default`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_default) used by the agent should have `read` access to a given node so that the DNS interface can be queried. Node rules are used to filter query results when reading from the catalog or retrieving information from the health endpoints. This allows for configurations where a token has access to a given service name, but only on an allowed subset of node names. Consul agents check tokens locally when health checks are registered and when Consul performs periodic [anti-entropy](/consul/docs/concept/consistency) syncs. These actions may required an ACL token to complete. Use the following methods to configure ACL tokens for registration events: -* Configure a global token in the [acl.tokens.default](/consul/docs/reference/agent#acl_tokens_default) parameter. +* Configure a global token in the [acl.tokens.default](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_default) parameter. This allows a single token to be used during all check registration operations. * Provide an ACL token with `service` and `check` definitions at registration time. This allows for greater flexibility and enables the use of multiple tokens on the same agent. @@ -809,7 +809,7 @@ service "admin" { Consul's DNS interface is affected by restrictions on service rules. If the -[`acl.tokens.default`](/consul/docs/reference/agent#acl_tokens_default) used by the agent does not have `read` access to a +[`acl.tokens.default`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_default) used by the agent does not have `read` access to a given service, then the DNS interface will return no records when queried for it. When reading from the catalog or retrieving information from the health endpoints, service rules are @@ -821,7 +821,7 @@ performs periodic [anti-entropy](/consul/docs/concept/consistency) syncs, which ACL token to complete. To accommodate this, Consul provides two methods of configuring ACL tokens to use for registration events: -1. Using the [acl.tokens.default](/consul/docs/reference/agent#acl_tokens_default) configuration +1. Using the [acl.tokens.default](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_default) configuration directive. This allows a single token to be configured globally and used during all service and check registration operations. 2. Providing an ACL token with service and check definitions at registration @@ -836,8 +836,8 @@ to use for registration events: access. In addition to ACLs, in Consul 0.9.0 and later, the agent must be configured with -[`enable_script_checks`](/consul/docs/reference/agent#enable_script_checks) or -[`enable_local_script_checks`](/consul/docs/reference/agent#enable_local_script_checks) +[`enable_script_checks`](/consul/docs/reference/agent/configuration-file/general#enable_script_checks) or +[`enable_local_script_checks`](/consul/docs/reference/agent/configuration-file/general#enable_local_script_checks) set to `true` in order to enable script checks. ### Reading Imported Services @@ -925,4 +925,4 @@ session "admin" { } ``` - \ No newline at end of file + diff --git a/website/content/docs/secure/acl/token/agent.mdx b/website/content/docs/secure/acl/token/agent.mdx index 5b1de3f2028..440d6b60022 100644 --- a/website/content/docs/secure/acl/token/agent.mdx +++ b/website/content/docs/secure/acl/token/agent.mdx @@ -13,7 +13,7 @@ This topic describes how to create a token that you can use to register an agent Consul agents must present a token linked to policies that grant the appropriate set of permissions in order to register into the catalog and to discover services and nodes in the catalog. -Specify the [`agent`](/consul/docs/reference/agent#acl_tokens_agent) token to the Consul agent so that it can present the token when it registers into the catalog. +Specify the [`agent`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_agent) token to the Consul agent so that it can present the token when it registers into the catalog. ### Node identities versus custom policies @@ -384,7 +384,7 @@ Configure the Consul agent to present the token by either specifying the token i ### Apply the token in a file -Specify the token in the [`acl.token.agent`](/consul/docs/reference/agent#acl_tokens_agent) field of the agent configuration file so that the agent can present it and register into the catalog on startup. +Specify the token in the [`acl.token.agent`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_agent) field of the agent configuration file so that the agent can present it and register into the catalog on startup. ```hcl acl = { diff --git a/website/content/docs/secure/acl/token/dns.mdx b/website/content/docs/secure/acl/token/dns.mdx index e977eac1c34..6b128a04f25 100644 --- a/website/content/docs/secure/acl/token/dns.mdx +++ b/website/content/docs/secure/acl/token/dns.mdx @@ -15,7 +15,7 @@ The Consul binary ships with a DNS server that you can use for service discovery A Consul agent must be configured with a token linked to policies that grant the appropriate set of permissions. -Specify the [`default`](/consul/docs/reference/agent#acl_tokens_default) token to the Consul agent to authorize the agent to respond to DNS queries. Refer to [DNS usage overview](/consul/docs/discover/dns) for details on configuring and using Consul DNS. +Specify the [`default`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_default) token to the Consul agent to authorize the agent to respond to DNS queries. Refer to [DNS usage overview](/consul/docs/discover/dns) for details on configuring and using Consul DNS. ## Requirements @@ -320,7 +320,7 @@ Configure the Consul agent with the token by either specifying the token in the ### Apply the token in a file -Specify the token in the [`default`](/consul/docs/reference/agent#acl_tokens_default) field of the agent configuration file so that the agent can present it and register into the catalog on startup. +Specify the token in the [`default`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_default) field of the agent configuration file so that the agent can present it and register into the catalog on startup. ```hcl acl = { @@ -335,7 +335,7 @@ acl = { ### Apply the token with a command -Set the `default` token using the [`acl.token.default`](/consul/docs/reference/agent#acl_tokens_default) command. The following command configures a running Consul agent token with the specified token. +Set the `default` token using the [`acl.token.default`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_default) command. The following command configures a running Consul agent token with the specified token. ```shell-session $ consul acl set-agent-token default diff --git a/website/content/docs/secure/acl/token/federation.mdx b/website/content/docs/secure/acl/token/federation.mdx index 828b0dbdbb2..78d2b8aa2ff 100644 --- a/website/content/docs/secure/acl/token/federation.mdx +++ b/website/content/docs/secure/acl/token/federation.mdx @@ -69,11 +69,11 @@ acl { -Refer to the [ACL stanza in the Agent](/consul/docs/reference/agent#acl) reference docs for more detailed descriptions of each attribute. +Refer to the [ACL stanza in the Agent](/consul/docs/reference/agent/configuration-file/acl) reference docs for more detailed descriptions of each attribute. -Most enterprise deployments have security requirements that prevent specifying tokens in configuration files. The configuration example also sets the `enable_token_persistence` attribute so that the token is stored to disk in the agent's [data directory](/consul/docs/reference/agent#_data_dir). Any changes to the token through the [Consul HTTP API](/consul/api-docs/agent#update-acl-tokens) persists to the same location, and the value in the configuration file is ignored. +Most enterprise deployments have security requirements that prevent specifying tokens in configuration files. The configuration example also sets the `enable_token_persistence` attribute so that the token is stored to disk in the agent's [data directory](/consul/docs/reference/agent/configuration-file/general#data_dir). Any changes to the token through the [Consul HTTP API](/consul/api-docs/agent#update-acl-tokens) persists to the same location, and the value in the configuration file is ignored. @@ -94,7 +94,7 @@ You must configure servers in secondary datacenters to point to the primary data Replication tokens require the following permissions: | Permission | Description | -| -------------------- | ----------------------------------- | +| -------------------- | ----------------------------------- | | `acl = "write"` | The permission allows you to replicate tokens. | | `operator = "write"` | This permission enables the `proxy-default` configuration entries to be replicated and enables CA certificate signing in the secondary datacenter. | | `policy = "read"` and `intentions = "read"` in the `service_prefix` field | These permissions enable `service-default` configuration entries, CA, and intention data to be replicated for all services. | @@ -134,7 +134,7 @@ namespace_prefix "" { policy = "read" intentions = "read" } -} +} ``` @@ -303,4 +303,4 @@ $ curl http://localhost:8500/v1/acl/replication?pretty } ``` -Notice `ReplicationType` is set to `tokens`. This means tokens, policies, and roles are replicated from the primary datacenter to the secondary one. You can call the `/v1/acl/replication` endpoint in the primary datacenter, but it does not return replication configuration. \ No newline at end of file +Notice `ReplicationType` is set to `tokens`. This means tokens, policies, and roles are replicated from the primary datacenter to the secondary one. You can call the `/v1/acl/replication` endpoint in the primary datacenter, but it does not return replication configuration. diff --git a/website/content/docs/secure/acl/token/index.mdx b/website/content/docs/secure/acl/token/index.mdx index d91e5aead51..f11c9ba4c28 100644 --- a/website/content/docs/secure/acl/token/index.mdx +++ b/website/content/docs/secure/acl/token/index.mdx @@ -17,7 +17,7 @@ Refer to the [ACL workflow overview](/consul/docs/secure/acl#workflow-overview) ## Create tokens -The ACLs administrator can use the HTTP API or CLI to create and link tokens to entities that enable permissions to resources. +The ACLs administrator can use the HTTP API or CLI to create and link tokens to entities that enable permissions to resources. @@ -116,7 +116,7 @@ Refer to [service definition reference](/consul/docs/reference/service) for more ### Configure agents with ACLs -You can configure Consul agents to hold several ACL tokens ([`acl.tokens`](#special-purpose-tokens)) to accommodate several use cases, such as bootstrapping the ACL system or accessing Consul under specific conditions. +You can configure Consul agents to hold several ACL tokens ([`acl.tokens`](#special-purpose-tokens)) to accommodate several use cases, such as bootstrapping the ACL system or accessing Consul under specific conditions. In the following example, the agent is configured to use a default token: @@ -142,7 +142,7 @@ acls { -Refer to the [agent configuration reference](/consul/docs/reference/agent#acl) for more information. +Refer to the [agent configuration reference](/consul/docs/reference/agent/configuration-file/acl) for more information. # Rotate token @@ -178,11 +178,11 @@ The following table describes a subset of special purpose tokens. Refer to [`acl | Token | Servers | Clients | Description | | ----------------------------- | ---------- | ---------- | ----------- | -| [`acl.tokens.agent`](/consul/docs/reference/agent#acl_tokens_agent) | `optional` | `optional` | Used for internal agent operations. Refer to [ACL agent token](#acl-agent-token) for details. | -| [`acl.tokens.agent_recovery`](/consul/docs/reference/agent#acl_tokens_agent_recovery) | `optional` | `optional` | Enables access to the [`agent/` HTTP API endpoint](/consul/api-docs/agent) when remote bearer token resolution fails.
Used for setting up the cluster and performing initial join operations.
Refer to [ACL agent recovery token](#acl-agent-recovery-token) for details. | -| [`acl.tokens.default`](/consul/docs/reference/agent#acl_tokens_default) | `optional` | `optional` | Specifies a default token to use for client requests if no token is supplied. This token is commonly configured with read-only access to services to enable DNS service discovery on agents. | -| [`acl.tokens.initial_management`](/consul/docs/reference/agent#acl_tokens_initial_management) | `optional` | `n/a` | Used to bootstrap the ACL system. Refer to [Initial management token](#initial-management-token). | -| [`acl.tokens.replication`](/consul/docs/reference/agent#acl_tokens_replication) | `optional` | `n/a` | Authorizes secondary datacenters to replicate data from the primary datacenter. | +| [`acl.tokens.agent`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_agent) | `optional` | `optional` | Used for internal agent operations. Refer to [ACL agent token](#acl-agent-token) for details. | +| [`acl.tokens.agent_recovery`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_agent_recovery) | `optional` | `optional` | Enables access to the [`agent/` HTTP API endpoint](/consul/api-docs/agent) when remote bearer token resolution fails.
Used for setting up the cluster and performing initial join operations.
Refer to [ACL agent recovery token](#acl-agent-recovery-token) for details. | +| [`acl.tokens.default`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_default) | `optional` | `optional` | Specifies a default token to use for client requests if no token is supplied. This token is commonly configured with read-only access to services to enable DNS service discovery on agents. | +| [`acl.tokens.initial_management`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_initial_management) | `optional` | `n/a` | Used to bootstrap the ACL system. Refer to [Initial management token](#initial-management-token). | +| [`acl.tokens.replication`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_replication) | `optional` | `n/a` | Authorizes secondary datacenters to replicate data from the primary datacenter. | You can use the [`/v1/agent/token` endpoint](/consul/api-docs/agent#update-acl-tokens) to create or update all reserved tokens except the `initial_management` token. @@ -192,13 +192,13 @@ Snapshots are artifacts created with the [snapshot API](/consul/api-docs/snapsho ### ACL agent token -The ACL agent token ([`acl.tokens.agent`](/consul/docs/reference/agent#acl_tokens_agent)) is a special token that Consul uses for internal agent operations. Consul does not use this token directly for any user-initiated operations like the default ACL token ([`acl.tokens.default`](/consul/docs/reference/agent#acl_tokens_default). However, if you do not define the ACL agent token, Consul will use the default ACL token for internal agent operations. Consul uses the ACL agent token for the following agent operations: +The ACL agent token ([`acl.tokens.agent`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_agent)) is a special token that Consul uses for internal agent operations. Consul does not use this token directly for any user-initiated operations like the default ACL token ([`acl.tokens.default`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_default). However, if you do not define the ACL agent token, Consul will use the default ACL token for internal agent operations. Consul uses the ACL agent token for the following agent operations: 1. Updating the agent's node entry using the [Catalog API](/consul/api-docs/catalog). This includes updating its node metadata, tagged addresses, and network coordinates. 2. Performing [anti-entropy](/consul/docs/concept/consistency) syncing, in particular reading the node metadata and services registered with the catalog. 3. Reading and writing the special `_rexec` section of the KV store when executing [`consul exec`](/consul/commands/exec) commands. -The following example is the minimum policy required to perform the internal agent operations for a node called `mynode`. The `service_prefix` policy needs read access to any services that can be registered on the agent. If [remote execution is disabled](/consul/docs/reference/agent#disable_remote_exec) (default behavior), you can omit the `key_prefix` policy. +The following example is the minimum policy required to perform the internal agent operations for a node called `mynode`. The `service_prefix` policy needs read access to any services that can be registered on the agent. If [remote execution is disabled](/consul/docs/reference/agent/configuration-file/general#disable_remote_exec) (default behavior), you can omit the `key_prefix` policy. @@ -236,13 +236,13 @@ You may update the anonymous token's description and policies, however you canno There are two ways for you to create an initial management token. 1. Bootstrap the ACL system with `consul acl bootstrap`. Consul automatically generates a token for you. Refer to [Bootstrap ACLs](/consul/docs/secure/acl/bootstrap) for more details. -1. Define an initial management token in the Consul agent configuration ([`acl.token.initial_management`](/consul/docs/reference/agent#acl_tokens_initial_management)). This method lets ACL administrators to bootstrap the ACL system with a known value. +1. Define an initial management token in the Consul agent configuration ([`acl.token.initial_management`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_initial_management)). This method lets ACL administrators to bootstrap the ACL system with a known value. Consul links the management token to the built-in [global management](/consul/docs/secure/acl/policy#global-management) policy. This token has unrestricted privileges to resources and APIs. ### ACL agent recovery token -You can use the ACL agent recovery token ([`acl.tokens.agent_recovery`](/consul/docs/reference/agent#acl_tokens_agent_recovery)) when your Consul servers are unavailable. The policy linked to the token is managed locally on the agent and does not require a token to be defined on the Consul servers. Once set, it implicitly has the following policy associated with it. +You can use the ACL agent recovery token ([`acl.tokens.agent_recovery`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_agent_recovery)) when your Consul servers are unavailable. The policy linked to the token is managed locally on the agent and does not require a token to be defined on the Consul servers. Once set, it implicitly has the following policy associated with it. ```hcl hideClipboard agent "" { diff --git a/website/content/docs/secure/acl/token/replication.mdx b/website/content/docs/secure/acl/token/replication.mdx index 1742fc3d9dd..55d6e01d861 100644 --- a/website/content/docs/secure/acl/token/replication.mdx +++ b/website/content/docs/secure/acl/token/replication.mdx @@ -12,7 +12,7 @@ This topic describes how to configure an ACL token for ACL replication between W ## Introduction Consul agents must present a token linked to policies that grant the appropriate set of permissions. -Specify the [`replication`](/consul/docs/reference/agent#acl_tokens_replication) token on each server in a non-primary datacenter. For hands-on instructions on how to configure ACL replication across datacenters, refer to [Enable ACLs in WAN federated datacenters](/consul/docs/secure/acl/token/replication). +Specify the [`replication`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_replication) token on each server in a non-primary datacenter. For hands-on instructions on how to configure ACL replication across datacenters, refer to [Enable ACLs in WAN federated datacenters](/consul/docs/secure/acl/token/replication). ## Requirements @@ -288,7 +288,7 @@ Configure the Consul agent with the token by either specifying the token in the ### Apply the token in a file -Specify the token in the [`replication`](/consul/docs/reference/agent#acl_tokens_replication) field of the agent configuration file so that the agent can present it and register into the catalog on startup. +Specify the token in the [`replication`](/consul/docs/reference/agent/configuration-file/acl#acl_tokens_replication) field of the agent configuration file so that the agent can present it and register into the catalog on startup. ```hcl acl = { diff --git a/website/content/docs/secure/encryption/gossip/enable/existing.mdx b/website/content/docs/secure/encryption/gossip/enable/existing.mdx index e9a5fa2928d..429db09b583 100644 --- a/website/content/docs/secure/encryption/gossip/enable/existing.mdx +++ b/website/content/docs/secure/encryption/gossip/enable/existing.mdx @@ -1,7 +1,7 @@ --- layout: docs page_title: Enable gossip encryption for existing datacenters -description: >- +description: >- Consul encrypts communication between agents in the LAN gossip pool. Learn how to generate a gossip encryption key and distribute it to agents in an existing Consul datacenter. --- @@ -20,7 +20,7 @@ The following steps describe the general workflow for enabling gossip encryption 1. Start the Consul agent with `encrypt_verify_outgoing` enabled. 1. Start the Consul agent with both `encrypt_verify_incoming` and `encrypt_verify_outgoing` enabled. -These steps are similar to enabling gossip encryption on new datacenters, but require you to configure the [`encrypt_verify_incoming`](/consul/docs/reference/agent#encrypt_verify_incoming) and [`encrypt_verify_outgoing`](/consul/docs/reference/agent#encrypt_verify_outgoing) agent configuration parameters. +These steps are similar to enabling gossip encryption on new datacenters, but require you to configure the [`encrypt_verify_incoming`](/consul/docs/reference/agent/configuration-file/encryption#encrypt_verify_incoming) and [`encrypt_verify_outgoing`](/consul/docs/reference/agent/configuration-file/encryption#encrypt_verify_outgoing) agent configuration parameters. ## Generate an encryption key @@ -33,7 +33,7 @@ pUqJrVyVRj5jsiYEkM/tFQYfWyJIv4s3XkvDwy7Cu5s= ## Disable verify encryption -Set the [`encrypt`](/consul/docs/reference/agent#_encrypt) key, and set `encrypt_verify_incoming` and `encrypt_verify_outgoing` to `false` in the agent configuration file. Then initiate a rolling update of all the agents with these new values. After this step, the agents will be able to decrypt gossip but will not yet be able to send encrypted traffic. +Set the [`encrypt`](/consul/docs/reference/agent/configuration-file/encryption#encrypt) key, and set `encrypt_verify_incoming` and `encrypt_verify_outgoing` to `false` in the agent configuration file. Then initiate a rolling update of all the agents with these new values. After this step, the agents will be able to decrypt gossip but will not yet be able to send encrypted traffic. @@ -138,4 +138,6 @@ encrypt_verify_outgoing = true -All the agents are configured to strictly enforce encrypted gossip. The default behavior of both `encrypt_verify_incoming` and `encrypt_verify_outgoing` is `true`. \ No newline at end of file +All the agents are configured to strictly enforce encrypted gossip. The default +behavior of both `encrypt_verify_incoming` and `encrypt_verify_outgoing` is +`true`. diff --git a/website/content/docs/secure/encryption/tls/enable/new/builtin.mdx b/website/content/docs/secure/encryption/tls/enable/new/builtin.mdx index eea38736093..de8426ac401 100644 --- a/website/content/docs/secure/encryption/tls/enable/new/builtin.mdx +++ b/website/content/docs/secure/encryption/tls/enable/new/builtin.mdx @@ -1,8 +1,8 @@ --- layout: docs page_title: Enable TLS encryption with built-in CA -description: >- - Learn how to create and distribute TLS certificates to server and client agents using Consul's built-in certificate authority (CA). +description: >- + Learn how to create and distribute TLS certificates to server and client agents using Consul's built-in certificate authority (CA). --- # Enable TLS encryption with built-in certificate authority @@ -88,10 +88,10 @@ $ consul tls cert create -server -dc dc1 -additional-dnsname="*.dc2.consul" Refer to the [`consul tls cert`](/consul/commands/tls/cert) CLI documentation to learn more about TLS certificate creation options. -To authenticate Consul servers, you must provide servers with a certificate that contains `server.dc1.consul` in the `Common Name` and in the `Subject Alternative Name`. If you enable [`verify_server_hostname`](/consul/docs/reference/agent#verify_server_hostname), only agents that provide such certificate are allowed to boot as a server. Without `verify_server_hostname = true`, an attacker could compromise a Consul client agent and restart the agent as a server in order to get access to all the data in your datacenter. These server certificates are special, and only servers should have them provisioned. +To authenticate Consul servers, you must provide servers with a certificate that contains `server.dc1.consul` in the `Common Name` and in the `Subject Alternative Name`. If you enable [`verify_server_hostname`](/consul/docs/reference/agent/configuration-file/tls#verify_server_hostname), only agents that provide such certificate are allowed to boot as a server. Without `verify_server_hostname = true`, an attacker could compromise a Consul client agent and restart the agent as a server in order to get access to all the data in your datacenter. These server certificates are special, and only servers should have them provisioned. - + Server keys, like the CA key, must be kept private. They effectively allow access to all Consul data. @@ -110,9 +110,9 @@ Repeat this process until all servers have these three files. ## Distribute client certificate -There are two methods for distributing client certificates: +There are two methods for distributing client certificates: -1. **Auto-encryption method.** We recommend the [auto-encryption method](/consul/docs/reference/agent#auto_encrypt) to alleviate the client certificate generation and distribution step for operators. This method uses the Connect CA to generate client certificates, which are automatically distributed to all Consul clients. +1. **Auto-encryption method.** We recommend the [auto-encryption method](/consul/docs/reference/agent/configuration-file/encryption#auto_encrypt) to alleviate the client certificate generation and distribution step for operators. This method uses the Connect CA to generate client certificates, which are automatically distributed to all Consul clients. 1. **Operator method.** Use this method if you need to use a third-party CA or need more fine-grained control over certificate management. @@ -155,7 +155,7 @@ auto_encrypt { -In addition to the `verify` settings, you need to enable `allow_tls`. The `verify` settings ensure that all communication between servers and clients is verified. When [`auto_encrypt`](/consul/docs/reference/agent#allow_tls) is enabled, `verify_outgoing` is enabled by default. +In addition to the `verify` settings, you need to enable `allow_tls`. The `verify` settings ensure that all communication between servers and clients is verified. When [`auto_encrypt`](/consul/docs/reference/agent/configuration-file/encryption#allow_tls) is enabled, `verify_outgoing` is enabled by default. @@ -186,7 +186,7 @@ key_file = "dc1-server-consul-0-key.pem" -Consul uses TLS to verify the authenticity of the servers with [`verify_outgoing`](/consul/docs/reference/agent#verify_outgoing) and [`verify_server_hostname`](/consul/docs/reference/agent#verify_server_hostname). You can also use TLS to also optionally verify client certificates when using [`verify_incoming`](/consul/docs/reference/agent#verify_incoming). +Consul uses TLS to verify the authenticity of the servers with [`verify_outgoing`](/consul/docs/reference/agent/configuration-file/tls#verify_outgoing) and [`verify_server_hostname`](/consul/docs/reference/agent/configuration-file/tls#verify_server_hostname). You can also use TLS to also optionally verify client certificates when using [`verify_incoming`](/consul/docs/reference/agent/configuration-file/tls#verify_incoming). @@ -280,4 +280,4 @@ Now that you have configured your servers and clients, you can start Consul. ```shell-session $ systemctl start consul -``` \ No newline at end of file +``` diff --git a/website/content/docs/secure/encryption/tls/enable/new/openssl.mdx b/website/content/docs/secure/encryption/tls/enable/new/openssl.mdx index 4d5d476b791..d9e42c82a90 100644 --- a/website/content/docs/secure/encryption/tls/enable/new/openssl.mdx +++ b/website/content/docs/secure/encryption/tls/enable/new/openssl.mdx @@ -2,7 +2,7 @@ layout: docs page_title: Enable TLS encryption with OpenSSL description: >- - Learn how to create and distribute TLS certificates to server and client agents using OpenSSL as your certificate authority (CA). + Learn how to create and distribute TLS certificates to server and client agents using OpenSSL as your certificate authority (CA). --- # Enable TLS encryption with OpenSSL @@ -31,9 +31,9 @@ You can create these certificates on any machine with `openssl`. The examples in ### Create server certificate signing requests -To authenticate Consul servers, servers are provided with a special certificate that contains `server.dc1.consul` in the `Common Name`. If you enable [`verify_server_hostname`](/consul/docs/reference/agent#verify_server_hostname), only agents that provide this certificate are allowed to boot as a server. Without `verify_server_hostname = true`, an attacker could compromise a Consul client agent and restart the agent as a server in order to get access to all the data in your datacenter. These server certificates are special and only servers should provision them. +To authenticate Consul servers, servers are provided with a special certificate that contains `server.dc1.consul` in the `Common Name`. If you enable [`verify_server_hostname`](/consul/docs/reference/agent/configuration-file/tls#verify_server_hostname), only agents that provide this certificate are allowed to boot as a server. Without `verify_server_hostname = true`, an attacker could compromise a Consul client agent and restart the agent as a server in order to get access to all the data in your datacenter. These server certificates are special and only servers should provision them. -Create a certificate signing request with the private key. The following command generates a CSR that has Common Name `server.dc1.consul`. Configure the CSR to use the [name of your datacenter](/consul/docs/reference/agent#_datacenter) and your [domain](/consul/docs/reference/agent#_domain). +Create a certificate signing request with the private key. The following command generates a CSR that has Common Name `server.dc1.consul`. Configure the CSR to use the [name of your datacenter](/consul/docs/reference/agent/configuration=file/general#datacenter) and your [domain](/consul/docs/reference/agent/configuration-file/dns#domain). ```shell-session $ openssl req -new -newkey rsa:2048 -nodes -keyout server1.dc1.consul.key -out server1.dc1.consul.csr -subj '/CN=server.dc1.consul' @@ -145,15 +145,15 @@ For Consul to use the certificate you created, copy them to the agents you want Every Consul agent needs 3 files to complete the configuration: -1. CA public certificate: defined by [`ca_file`](/consul/docs/reference/agent#ca_file) parameter is used to verify the identity of the other nodes. For these instructions, the CA public certificate is named `consul-agent-ca.pem`. -1. Consul agent public certificate: defined by [`cert_file`](/consul/docs/reference/agent#cert_file) parameter. -1. Consul agent private key: defined by [`key_file`](/consul/docs/reference/agent#key_file) parameter. +1. CA public certificate: defined by [`ca_file`](/consul/docs/reference/agent/configuration-file/tls#ca_file) parameter is used to verify the identity of the other nodes. For these instructions, the CA public certificate is named `consul-agent-ca.pem`. +1. Consul agent public certificate: defined by [`cert_file`](/consul/docs/reference/agent/configuration-file/tls#cert_file) parameter. +1. Consul agent private key: defined by [`key_file`](/consul/docs/reference/agent/configuration-file/tls#key_file) parameter. Consul looks for these certificate files in the start directory. If you stored the files there, you can add them to the agent configuration file by their name only. When the certificate files are not located in the start directory, you must specify the path in the agent configuration file. ### Configure servers -You can add the suggested parameter as a separate file in the [`-config-dir`](/consul/docs/reference/agent#_config_dir) directory from where Consul loads the configuration. This approach helps you keep configurations separate, but you can also include the options in a single configuration file. +You can add the suggested parameter as a separate file in the [`-config-dir`](/consul/docs/fundamentals/agent#start-the-consul-agent) directory from where Consul loads the configuration. This approach helps you keep configurations separate, but you can also include the options in a single configuration file. The following example of an agent TLS configuration for Consul servers shows the distributed certificate files. @@ -189,7 +189,7 @@ ports { -Consul can use TLS to verify the authenticity of the servers with [`verify_outgoing`](/consul/docs/reference/agent#verify_outgoing) and [`verify_server_hostname`](/consul/docs/reference/agent#verify_server_hostname). It can also optionally verify client certificates when using [`verify_incoming`](/consul/docs/reference/agent#verify_incoming). +Consul can use TLS to verify the authenticity of the servers with [`verify_outgoing`](/consul/docs/reference/agent/configuration-file/tls#verify_outgoing) and [`verify_server_hostname`](/consul/docs/reference/agent/configuration-file/tls#verify_server_hostname). It can also optionally verify client certificates when using [`verify_incoming`](/consul/docs/reference/agent/configuration-file/tls#verify_incoming). The example configuration also disables the HTTP port to ensure that there is only encrypted communication. With HTTPS enabled, all CLI, API, and UI communication needs to verify their identity with a certificate. This affects built-in tooling like `consul members` and the UI. @@ -396,9 +396,9 @@ writing new private key to 'cli.client.dc1.consul.key' ### Select an interface to bind to -Depending on your setup, you might need to change to which interface you are binding. By default, the UI binds to `127.0.0.1`. +Depending on your setup, you might need to change to which interface you are binding. By default, the UI binds to `127.0.0.1`. -Use [`addresses.https`](/consul/docs/reference/agent#https) or [`client_addr`](/consul/docs/reference/agent#client_addr) to update the interface. Be mindful when updating these values because it also impacts the DNS server. +Use [`addresses.https`](/consul/docs/reference/agent/configuration-file/general#https) or [`client_addr`](/consul/docs/reference/agent/configuration-file/general#client_addr) to update the interface. Be mindful when updating these values because it also impacts the DNS server. We recommend binding to `0.0.0.0`. @@ -428,7 +428,7 @@ disable_remote_exec = true -Since your Consul agent is now available to the network, verify that [`enable_script_checks`](/consul/docs/reference/agent#_enable_script_checks) is `false` and [`disable_remote_exec`](/consul/docs/reference/agent#disable_remote_exec) is `true`. +Since your Consul agent is now available to the network, verify that [`enable_script_checks`](/consul/docs/reference/agent/configuration-file/general#enable_script_checks) is `false` and [`disable_remote_exec`](/consul/docs/reference/agent/configuration-file/general#disable_remote_exec) is `true`. ## Access the Consul UI @@ -449,7 +449,7 @@ HTTP/2 200 ## ... ``` -### Enable `verify_incoming_rpc` and disable `verify_incoming` +### Enable `verify_incoming_rpc` and disable `verify_incoming` When `verify_incoming` is enabled, Consul requires that all incoming connections make use of TLS and that the client provides a certificate signed by a CA from the `ca_file` or `ca_path`. This restriction applies to both server RPC and to the HTTPS API. Because the browser does not present a certificate signed by your CA, you cannot access the UI. If you `curl` your HTTPS UI, it returns an error. diff --git a/website/content/docs/secure/security-model/core.mdx b/website/content/docs/secure/security-model/core.mdx index 5deb1453b77..c3520983b18 100644 --- a/website/content/docs/secure/security-model/core.mdx +++ b/website/content/docs/secure/security-model/core.mdx @@ -43,25 +43,25 @@ Consul's security model is applicable only if all parts of the system are runnin - **mTLS** - Mutual authentication of both the TLS server and client x509 certificates prevents internal abuse through unauthorized access to Consul agents within the cluster. - - [`tls.defaults.verify_incoming`](/consul/docs/reference/agent#tls_defaults_verify_incoming) - By default this is false, and should almost always be set to true to require TLS verification for incoming client connections. This applies to the internal RPC, HTTPS and gRPC APIs. + - [`tls.defaults.verify_incoming`](/consul/docs/reference/agent/configuration-file/tls#tls_defaults_verify_incoming) - By default this is false, and should almost always be set to true to require TLS verification for incoming client connections. This applies to the internal RPC, HTTPS and gRPC APIs. - - [`tls.https.verify_incoming`](/consul/docs/reference/agent#tls_https_verify_incoming) - By default this is false, and should be set to true to require clients to provide a valid TLS certificate when the Consul HTTPS API is enabled. TLS for the API may be not be necessary if it is exclusively served over a loopback interface such as `localhost`. + - [`tls.https.verify_incoming`](/consul/docs/reference/agent/configuration-file/tls#tls_https_verify_incoming) - By default this is false, and should be set to true to require clients to provide a valid TLS certificate when the Consul HTTPS API is enabled. TLS for the API may be not be necessary if it is exclusively served over a loopback interface such as `localhost`. - - [`tls.internal_rpc.verify_incoming`](/consul/docs/reference/agent#tls_internal_rpc_verify_incoming) - By default this is false, and should almost always be set to true to require clients to provide a valid TLS certificate for Consul agent RPCs. + - [`tls.internal_rpc.verify_incoming`](/consul/docs/reference/agent/configuration-file/tls#tls_internal_rpc_verify_incoming) - By default this is false, and should almost always be set to true to require clients to provide a valid TLS certificate for Consul agent RPCs. - - [`tls.grpc.verify_incoming`](/consul/docs/reference/agent#tls_grpc_verify_incoming) - By default this is false, and should be set to true to require clients to provide a valid TLS certificate when the Consul gRPC API is enabled. TLS for the API may be not be necessary if it is exclusively served over a loopback interface such as `localhost`. + - [`tls.grpc.verify_incoming`](/consul/docs/reference/agent/configuration-file/tls#tls_grpc_verify_incoming) - By default this is false, and should be set to true to require clients to provide a valid TLS certificate when the Consul gRPC API is enabled. TLS for the API may be not be necessary if it is exclusively served over a loopback interface such as `localhost`. - - [`tls.internal_rpc.verify_outgoing`](/consul/docs/reference/agent#tls_internal_rpc_verify_outgoing) - By default this is false, and should be set to true to require TLS for outgoing connections from server or client agents. Servers that specify `verify_outgoing = true` will always talk to other servers over TLS, but they still accept non-TLS connections to allow for a transition of all clients to TLS. Currently the only way to enforce that no client can communicate with a server unencrypted is to also enable `verify_incoming` which requires client certificates too. + - [`tls.internal_rpc.verify_outgoing`](/consul/docs/reference/agent/configuration-file/tls#tls_internal_rpc_verify_outgoing) - By default this is false, and should be set to true to require TLS for outgoing connections from server or client agents. Servers that specify `verify_outgoing = true` will always talk to other servers over TLS, but they still accept non-TLS connections to allow for a transition of all clients to TLS. Currently the only way to enforce that no client can communicate with a server unencrypted is to also enable `verify_incoming` which requires client certificates too. - - [`enable_agent_tls_for_checks`](/consul/docs/reference/agent#enable_agent_tls_for_checks) - By default this is false, and should almost always be set to true to require mTLS to set up the client for HTTP or gRPC health checks. This was added in Consul 1.0.1. + - [`enable_agent_tls_for_checks`](/consul/docs/reference/agent/configuration-file/general#enable_agent_tls_for_checks) - By default this is false, and should almost always be set to true to require mTLS to set up the client for HTTP or gRPC health checks. This was added in Consul 1.0.1. - - [`tls.internal_rpc.verify_server_hostname`](/consul/docs/reference/agent#tls_internal_rpc_verify_server_hostname) - By default this is false, and should be set to true to require that the TLS certificate presented by the servers matches `server..` hostname for outgoing TLS connections. The default configuration does not verify the hostname of the certificate, only that it is signed by a trusted CA. This setting is critical to prevent a compromised client agent from being restarted as a server and having all cluster state including all ACL tokens and service mesh CA root keys replicated to it. This setting was introduced in 0.5.1. From version 0.5.1 to 1.4.0 we documented that `verify_server_hostname` being true implied verify_outgoing however due to a bug this was not the case so setting only `verify_server_hostname` results in plaintext communication between client and server. See [CVE-2018-19653](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-19653) for more details. This is fixed in 1.4.1. + - [`tls.internal_rpc.verify_server_hostname`](/consul/docs/reference/agen/configuration-file/tls#tls_internal_rpc_verify_server_hostname) - By default this is false, and should be set to true to require that the TLS certificate presented by the servers matches `server..` hostname for outgoing TLS connections. The default configuration does not verify the hostname of the certificate, only that it is signed by a trusted CA. This setting is critical to prevent a compromised client agent from being restarted as a server and having all cluster state including all ACL tokens and service mesh CA root keys replicated to it. This setting was introduced in 0.5.1. From version 0.5.1 to 1.4.0 we documented that `verify_server_hostname` being true implied verify_outgoing however due to a bug this was not the case so setting only `verify_server_hostname` results in plaintext communication between client and server. See [CVE-2018-19653](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-19653) for more details. This is fixed in 1.4.1. - - [`auto_encrypt`](/consul/docs/reference/agent#auto_encrypt) - Enables automated TLS certificate distribution for client agent RPC communication using the service mesh CA. Using this configuration a [`ca_file`](/consul/docs/reference/agent#tls_defaults_ca_file) and ACL token would still need to be distributed to client agents. + - [`auto_encrypt`](/consul/docs/reference/agent/configuration-file/encryption#auto_encrypt) - Enables automated TLS certificate distribution for client agent RPC communication using the service mesh CA. Using this configuration a [`ca_file`](/consul/docs/reference/agent/configuration-file/tls#tls_defaults_ca_file) and ACL token would still need to be distributed to client agents. - - [`allow_tls`](/consul/docs/reference/agent#allow_tls) - By default this is false, and should be set to true on server agents to allow certificates to be automatically generated and distributed from the service mesh CA to client agents. + - [`allow_tls`](/consul/docs/reference/agent/configuration-file/auto-encrypt#allow_tls) - By default this is false, and should be set to true on server agents to allow certificates to be automatically generated and distributed from the service mesh CA to client agents. - - [`tls`](/consul/docs/reference/agent#tls) - By default this is false, and should be set to true on client agents to automatically request a client TLS certificate from the server's service mesh CA. + - [`tls`](/consul/docs/reference/agent/configuration-file/tls) - By default this is false, and should be set to true on client agents to automatically request a client TLS certificate from the server's service mesh CA. @@ -102,14 +102,14 @@ Consul's security model is applicable only if all parts of the system are runnin ``` - + - The example client agent TLS configuration sets [`verify_incoming`](/consul/docs/reference/agent#tls_defaults_verify_incoming) to false which assumes all incoming traffic is restricted to `localhost`. The primary benefit for this configuration is to avoid provisioning client TLS certificates (in addition to ACL tokens) for all tools or applications using the local Consul agent. In this case, ACLs should be enabled to provide authorization and only ACL tokens would need to be distributed. + The example client agent TLS configuration sets [`verify_incoming`](/consul/docs/reference/agent/configuration-file/tls#tls_defaults_verify_incoming) to false which assumes all incoming traffic is restricted to `localhost`. The primary benefit for this configuration is to avoid provisioning client TLS certificates (in addition to ACL tokens) for all tools or applications using the local Consul agent. In this case, ACLs should be enabled to provide authorization and only ACL tokens would need to be distributed. -- **ACLs** - The access control list (ACL) system provides a security mechanism for Consul administrators to grant capabilities tied to an individual human, or machine operator identity. To ultimately secure the ACL system, administrators should configure the [`default_policy`](/consul/docs/reference/agent#acl_default_policy) to "deny". +- **ACLs** - The access control list (ACL) system provides a security mechanism for Consul administrators to grant capabilities tied to an individual human, or machine operator identity. To ultimately secure the ACL system, administrators should configure the [`default_policy`](/consul/docs/reference/agent/configuration-file/acl#acl_default_policy) to "deny". The [ACL system](/consul/docs/secure/acl) is comprised of five major components: @@ -127,9 +127,9 @@ Consul's security model is applicable only if all parts of the system are runnin Two optional gossip encryption options enable Consul servers without gossip encryption to safely upgrade. After upgrading, the verification options should be enabled, or removed to set them to their default state: - - [`encrypt_verify_incoming`](/consul/docs/reference/agent#encrypt_verify_incoming) - By default this is true to enforce encryption on _incoming_ gossip communications. + - [`encrypt_verify_incoming`](/consul/docs/reference/agent/configuration-file/encryption#encrypt_verify_incoming) - By default this is true to enforce encryption on _incoming_ gossip communications. - - [`encrypt_verify_outgoing`](/consul/docs/reference/agent#encrypt_verify_outgoing) - By default this is true to enforce encryption on _outgoing_ gossip communications. + - [`encrypt_verify_outgoing`](/consul/docs/reference/agent/configuration-file/encryption#encrypt_verify_outgoing) - By default this is true to enforce encryption on _outgoing_ gossip communications. - **Namespaces** - Read and write operations should be scoped to logical namespaces to restrict access to Consul components within a multi-tenant environment. Furthermore, this feature can be used to enable a self-service approach to Consul ACL administration for teams within a scoped namespace. @@ -153,13 +153,13 @@ Consul's security model is applicable only if all parts of the system are runnin - **Linux security modules** - Use of security modules that can be directly integrated into operating systems such as AppArmor, SElinux, and Seccomp on Consul agent hosts. -- **Customize TLS settings** - TLS settings such as the [available cipher suites](/consul/docs/reference/agent#tls_defaults_tls_cipher_suites), should be tuned to fit the needs of your environment. +- **Customize TLS settings** - TLS settings such as the [available cipher suites](/consul/docs/reference/agent/configuration-file/tls#tls_defaults_tls_cipher_suites), should be tuned to fit the needs of your environment. - - [`tls_min_version`](/consul/docs/reference/agent#tls_defaults_tls_min_version) - Used to specify the minimum TLS version to use. + - [`tls_min_version`](/consul/docs/reference/agent/configuration-file/tls#tls_defaults_tls_min_version) - Used to specify the minimum TLS version to use. - - [`tls_cipher_suites`](/consul/docs/reference/agent#tls_defaults_tls_cipher_suites) - Used to specify which TLS cipher suites are allowed. + - [`tls_cipher_suites`](/consul/docs/reference/agent/configuration-file/tls#tls_defaults_tls_cipher_suites) - Used to specify which TLS cipher suites are allowed. -- **Customize HTTP response headers** - Additional security headers, such as [`X-XSS-Protection`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-XSS-Protection), can be [configured](/consul/docs/reference/agent#response_headers) for HTTP API responses. +- **Customize HTTP response headers** - Additional security headers, such as [`X-XSS-Protection`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-XSS-Protection), can be [configured](/consul/docs/reference/agent/configuration-file/general#response_headers) for HTTP API responses. ```hcl http_config { @@ -171,21 +171,21 @@ Consul's security model is applicable only if all parts of the system are runnin - **Customize default limits** - Consul has builtin features with default connection limits that should be tuned to fit your environment. - - [`http_max_conns_per_client`](/consul/docs/reference/agent#http_max_conns_per_client) - Used to limit concurrent access from a single client to the HTTP(S) endpoint on Consul agents. + - [`http_max_conns_per_client`](/consul/docs/reference/agent/configuration-file/general#http_max_conns_per_client) - Used to limit concurrent access from a single client to the HTTP(S) endpoint on Consul agents. - - [`https_handshake_timeout`](/consul/docs/reference/agent#https_handshake_timeout) - Used to timeout TLS connection for the HTTP(S) endpoint for Consul agents. + - [`https_handshake_timeout`](/consul/docs/reference/agent/configuration-file/general#https_handshake_timeout) - Used to timeout TLS connection for the HTTP(S) endpoint for Consul agents. - - [`rpc_handshake_timeout`](/consul/docs/reference/agent#rpc_handshake_timeout) - Used to timeout TLS connections for the RPC endpoint for Consul agents. + - [`rpc_handshake_timeout`](/consul/docs/reference/agent/configuration-file/general#rpc_handshake_timeout) - Used to timeout TLS connections for the RPC endpoint for Consul agents. - - [`rpc_max_conns_per_client`](/consul/docs/reference/agent#rpc_max_conns_per_client) - Used to limit concurrent access from a single client to the RPC endpoint on Consul agents. + - [`rpc_max_conns_per_client`](/consul/docs/reference/agent/configuration-file/general#rpc_max_conns_per_client) - Used to limit concurrent access from a single client to the RPC endpoint on Consul agents. - - [`rpc_rate`](/consul/docs/reference/agent#rpc_rate) - Disabled by default, this is used to limit (requests/second) for client agents making RPC calls to server agents. + - [`rpc_rate`](/consul/docs/reference/agent/configuration-file/general#rpc_rate) - Disabled by default, this is used to limit (requests/second) for client agents making RPC calls to server agents. - - [`rpc_max_burst`](/consul/docs/reference/agent#rpc_max_burst) - Used as the token bucket size for client agents making RPC calls to server agents. + - [`rpc_max_burst`](/consul/docs/reference/agent/configuration-file/general#rpc_max_burst) - Used as the token bucket size for client agents making RPC calls to server agents. - - [`kv_max_value_size`](/consul/docs/reference/agent#kv_max_value_size) - Used to configure the max number of bytes in a key-value API request. + - [`kv_max_value_size`](/consul/docs/reference/agent/configuration-file/general#kv_max_value_size) - Used to configure the max number of bytes in a key-value API request. - - [`txn_max_req_len`](/consul/docs/reference/agent#txn_max_req_len) - Used to configure the max number of bytes in a transaction API request. + - [`txn_max_req_len`](/consul/docs/reference/agent/configuration-file/general#txn_max_req_len) - Used to configure the max number of bytes in a transaction API request. - **Secure UI access** - Access to Consul's builtin UI can be secured in various ways: @@ -195,7 +195,7 @@ Consul's security model is applicable only if all parts of the system are runnin - **ACL** - ACLs with a default deny policy enables safer UI access by preventing unauthorized access to sensitive components within the cluster. Refer to [ACLs](/consul/docs/secure/acl) documentation to learn more about ACLs and how to [bootstrap](/consul/docs/secure/acl/bootstrap) the ACLs system. - - **Restrict HTTP writes** - Using the [`allow_write_http_from`](/consul/docs/reference/agent#allow_write_http_from) configuration option to restrict write access for agent endpoints to hosts on the specified list of CIDRs. + - **Restrict HTTP writes** - Using the [`allow_write_http_from`](/consul/docs/reference/agent/configuration-file/general#allow_write_http_from) configuration option to restrict write access for agent endpoints to hosts on the specified list of CIDRs. @@ -204,7 +204,7 @@ Consul's security model is applicable only if all parts of the system are runnin allow_write_http_from = ["127.0.0.0/8"] } ``` - + ## Threat model