Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/reference/administering.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,5 @@ cluster.

--

include::ccr/index.asciidoc[]
include::administering/backup-cluster.asciidoc[]
5 changes: 3 additions & 2 deletions docs/reference/ccr/auto-follow.asciidoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[role="xpack"]
[testenv="platinum"]
[[ccr-auto-follow]]
=== Automatically following indices
==== Automatically following indices

In time series use cases where you want to follow new indices that are
periodically created (such as daily Beats indices), manually configuring follower
Expand All @@ -10,7 +10,8 @@ functionality in {ccr} is aimed at easing this burden. With the auto-follow
functionality, you can specify that new indices in a remote cluster that have a
name that matches a pattern are automatically followed.

==== Managing auto-follow patterns
[[ccr-manage-auto-follow-patterns]]
===== Managing auto-follow patterns

You can add a new auto-follow pattern configuration with the
{ref}/ccr-put-auto-follow-pattern.html[create auto-follow pattern API]. When you create
Expand Down
21 changes: 10 additions & 11 deletions docs/reference/ccr/getting-started.asciidoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[role="xpack"]
[testenv="platinum"]
[[ccr-getting-started]]
== Getting started with {ccr}
=== Getting started with {ccr}

This getting-started guide for {ccr} shows you how to:

Expand All @@ -12,9 +12,9 @@ This getting-started guide for {ccr} shows you how to:
a leader index
* <<ccr-getting-started-auto-follow,Automatically create follower indices>>

[float]

[[ccr-getting-started-before-you-begin]]
=== Before you begin
==== Before you begin
. {stack-gs}/get-started-elastic-stack.html#install-elasticsearch[Install {es}]
on your local and remote clusters.

Expand Down Expand Up @@ -81,9 +81,9 @@ cluster update settings API, you will also need a user with the `all` cluster
privilege.
--

[float]

[[ccr-getting-started-remote-cluster]]
=== Connecting to a remote cluster
==== Connecting to a remote cluster

The {ccr} features require that you
{ref}/modules-remote-clusters.html[connect your local cluster to a remote
Expand Down Expand Up @@ -157,9 +157,8 @@ Alternatively, you can manage remote clusters on the
image::images/remote-clusters.jpg["The Remote Clusters page in {kib}"]


[float]
[[ccr-getting-started-leader-index]]
=== Creating a leader index
==== Creating a leader index

In the following example, we will create a leader index in the remote cluster:

Expand Down Expand Up @@ -203,9 +202,9 @@ PUT /server-metrics
// CONSOLE
// TEST[continued]

[float]

[[ccr-getting-started-follower-index]]
=== Creating a follower index
==== Creating a follower index

Follower indices are created with the {ref}/ccr-put-follow.html[create follower
API]. When you create a follower index, you must reference the
Expand Down Expand Up @@ -263,9 +262,9 @@ POST /server-metrics-copy/_ccr/unfollow

//////////////////////////

[float]

[[ccr-getting-started-auto-follow]]
=== Automatically create follower indices
==== Automatically create follower indices

The <<ccr-auto-follow,auto-follow>> feature in {ccr} helps for time series use
cases where you want to follow new indices that are periodically created in the
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/ccr/index.asciidoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[role="xpack"]
[testenv="platinum"]
[[xpack-ccr]]
= {ccr-cap}
== {ccr-cap}

[partintro]
--
Expand Down
30 changes: 15 additions & 15 deletions docs/reference/ccr/overview.asciidoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[role="xpack"]
[testenv="platinum"]
[[ccr-overview]]
== Overview
=== Overview


{ccr-cap} is done on an index-by-index basis. Replication is
Expand All @@ -17,8 +17,8 @@ Replication is pull-based. This means that replication is driven by the
follower index. This simplifies state management on the leader index and means
that {ccr} does not interfere with indexing on the leader index.

[float]
=== Configuring replication
[[ccr-configure-replication]]
==== Configuring replication

Replication can be configured in two ways:

Expand Down Expand Up @@ -67,8 +67,8 @@ POST /follower_index/_ccr/pause_follow

//////////////////////////

[float]
=== The mechanics of replication
[[replication-mechanics]]
==== The mechanics of replication

While replication is managed at the index level, replication is performed at the
shard level. When a follower index is created, it is automatically
Expand Down Expand Up @@ -130,17 +130,17 @@ closing itself, applying the settings update, and then re-opening itself. The
follower index will be unavailable for reads and not replicating writes
during this cycle.

[float]
=== Inspecting the progress of replication
[[replication-progress]]
==== Inspecting the progress of replication

You can inspect the progress of replication at the shard level with the
{ref}/ccr-get-follow-stats.html[get follower stats API]. This API gives you
insight into the read and writes managed by the follower shard task. It also
reports read exceptions that can be retried and fatal exceptions that require
user intervention.

[float]
=== Pausing and resuming replication
[[pause-resume-replication]]
==== Pausing and resuming replication

You can pause replication with the
{ref}/ccr-post-pause-follow.html[pause follower API] and then later resume
Expand All @@ -149,8 +149,8 @@ Using these APIs in tandem enables you to adjust the read and write parameters
on the follower shard task if your initial configuration is not suitable for
your use case.

[float]
=== Leader index retaining operations for replication
[[retain-replication-ops]]
==== Leader index retaining operations for replication

If the follower is unable to replicate operations from a leader for a period of
time, the following process can fail due to the leader lacking a complete history
Expand Down Expand Up @@ -180,8 +180,8 @@ the lease expires. It is valuable to have monitoring in place to detect a follow
replication issue prior to the lease expiring so that the problem can be remedied
before the follower falls fatally behind.

[float]
=== Remedying a follower that has fallen behind
[[remedy-fallen-follower]]
==== Remedying a follower that has fallen behind

If a follower falls sufficiently behind a leader that it can no longer replicate
operations this can be detected in {kib} or by using the
Expand Down Expand Up @@ -211,8 +211,8 @@ segment files are deleted on the follower cluster. The
files from the leader again. After the follower index initializes, the
following process starts again.

[float]
=== Terminating replication
[[terminate-replication]]
==== Terminating replication

You can terminate replication with the
{ref}/ccr-post-unfollow.html[unfollow API]. This API converts a follower index
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/ccr/remote-recovery.asciidoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[role="xpack"]
[testenv="platinum"]
[[remote-recovery]]
== Remote recovery
=== Remote recovery

When you create a follower index, you cannot use it until it is fully initialized.
The _remote recovery_ process builds a new copy of a shard on a follower node by
Expand Down
14 changes: 7 additions & 7 deletions docs/reference/ccr/requirements.asciidoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[role="xpack"]
[testenv="platinum"]
[[ccr-requirements]]
=== Requirements for leader indices
==== Requirements for leader indices

{ccr-cap} works by replaying the history of individual write
operations that were performed on the shards of the leader index. This means that the
Expand All @@ -22,9 +22,9 @@ existing data that you want to replicate from another cluster, you must
{ref}/docs-reindex.html[reindex] your data into a new index with soft deletes
enabled.

[float]

[[ccr-overview-soft-deletes]]
==== Soft delete settings
===== Soft delete settings

`index.soft_deletes.enabled`::

Expand All @@ -42,9 +42,9 @@ The default value is `12h`.

For more information about index settings, see {ref}/index-modules.html[Index modules].

[float]

[[ccr-overview-beats]]
==== Setting soft deletes on indices created by APM Server or Beats
===== Setting soft deletes on indices created by APM Server or Beats

If you want to replicate indices created by APM Server or Beats, and are
allowing APM Server or Beats to manage index templates, you need to configure
Expand All @@ -63,9 +63,9 @@ For additional information on controlling the index templates managed by APM
Server or Beats, see the relevant documentation on loading the Elasticsearch
index template.

[float]

[[ccr-overview-logstash]]
==== Setting soft deletes on indices created by Logstash
===== Setting soft deletes on indices created by Logstash

If you want to replicate indices created by Logstash, and are using Logstash to
manage index templates, you need to configure soft deletes on a custom Logstash
Expand Down
10 changes: 5 additions & 5 deletions docs/reference/ccr/upgrading.asciidoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[role="xpack"]
[testenv="platinum"]
[[ccr-upgrading]]
== Upgrading clusters
=== Upgrading clusters

Clusters that are actively using {ccr} require a careful approach to upgrades.
Otherwise index following may fail during a rolling upgrade, because of the
Expand All @@ -17,8 +17,8 @@ following reasons:
Rolling upgrading clusters with {ccr} is different in case of uni-directional
index following and bi-directional index following.

[float]
=== Uni-directional index following
[[uni-directional-index-following]]
==== Uni-directional index following

In a uni-directional setup between two clusters, one cluster contains only
leader indices, and the other cluster contains only follower indices following
Expand All @@ -35,8 +35,8 @@ cluster B that follows indices in cluster A and cluster C that follows
indices in cluster B. In this case the cluster C should be upgraded first,
then cluster B and finally cluster A.

[float]
=== Bi-directional index following
[[bi-directional-index-following]]
==== Bi-directional index following

In a bi-directional setup between two clusters, each cluster contains both
leader and follower indices.
Expand Down