Skip to content

Commit 94ee8c7

Browse files
dakronedakrone
authored
[8.4] Fix renaming data streams with CCR replication (#88875) (#88995)
This commit fixes the situation where a user wants to use CCR to replicate indices that are part of a data stream while renaming the data stream. For example, assume a user has an auto-follow request that looks like this: ``` PUT /_ccr/auto_follow/my-auto-follow-pattern { "remote_cluster" : "other-cluster", "leader_index_patterns" : ["logs-*"], "follow_index_pattern" : "{{leader_index}}_copy" } ``` And then the data stream `logs-mysql-error` was created, creating the backing index `.ds-logs-mysql-error-2022-07-29-000001`. Prior to this commit, replicating this data stream means that the backing index would be renamed to `.ds-logs-mysql-error-2022-07-29-000001_copy` and the data stream would *not* be renamed. This caused a check to trip in `TransportPutLifecycleAction` asserting that a backing index was not renamed for a data stream during following. After this commit, there are a couple of changes: First, the data stream will also be renamed. This means that the `logs-mysql-error` becomes `logs-mysql-error_copy` when created on the follower cluster. Because of the way that CCR works, this means we need to support renaming a data stream for a regular "create follower" request, so a new parameter has been added: `data_stream_name`. It works like this: ``` PUT /mynewindex/_ccr/follow { "remote_cluster": "other-cluster", "leader_index": "myotherindex", "data_stream_name": "new_ds" } ``` Second, the backing index for a data stream must be renamed in a way that does not break the parsing of a data stream backing pattern, whereas previously the index `.ds-logs-mysql-error-2022-07-29-000001` would be renamed to `.ds-logs-mysql-error-2022-07-29-000001_copy` (an illegal name since it doesn't end with the rollover digit), after this commit it will be renamed to `.ds-logs-mysql-error_copy-2022-07-29-000001` to match the renamed data stream. This means that for the given `follow_index_pattern` of `{{leader_index}}_copy` the index changes look like: | Leader Cluster | Follower Cluster | |--------------|-----------| | `logs-mysql-error` (data stream) | `logs-mysql-error_copy` (data stream) | | `.ds-logs-mysql-error-2022-07-29-000001` | `.ds-logs-mysql-error_copy-2022-07-29-000001` | Which internally means the auto-follow request turned into the create follower request of: ``` PUT /.ds-logs-mysql-error_copy-2022-07-29-000001/_ccr/follow { "remote_cluster": "other-cluster", "leader_index": ".ds-logs-mysql-error-2022-07-29-000001", "data_stream_name": "logs-mysql-error_copy" } ``` Relates to #84940 (cherry-picked the commit for a test) Relates to #61993 (where data stream support was first introduced for CCR) Resolves #81751
1 parent 188d56a commit 94ee8c7

File tree

14 files changed

+753
-68
lines changed

14 files changed

+753
-68
lines changed

‎docs/changelog/88875.yaml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
pr: 88875
2+
summary: Fix renaming data streams with CCR replication
3+
area: "Data streams"
4+
type: bug
5+
issues:
6+
- 81751

‎docs/reference/ccr/apis/auto-follow/put-auto-follow-pattern.asciidoc‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -85,11 +85,14 @@ the new patterns.
8585
more `leader_index_patterns` and one or more `leader_index_exclusion_patterns` won't be followed.
8686

8787
`follow_index_pattern`::
88-
(Optional, string) The name of follower index. The template `{{leader_index}}`
89-
can be used to derive the name of the follower index from the name of the
90-
leader index. When following a data stream, use `{{leader_index}}`; {ccr-init}
91-
does not support changes to the names of a follower data stream's backing
92-
indices.
88+
(Optional, string) The name of follower index. The template `{{leader_index}}` can be used to
89+
derive the name of the follower index from the name of the leader index. When following a data
90+
stream, the `follow_index_pattern` will be used for renaming not only the leader index, but also
91+
the data stream containing the leader index. For example, a data stream called
92+
`logs-mysql-default` with a backing index of `.ds-logs-mysql-default-2022-01-01-000001` and a
93+
`follow_index_pattern` of `{{leader_index}}_copy` will replicate the data stream as
94+
`logs-mysql-default_copy` and the backing index as
95+
`.ds-logs-mysql-default_copy-2022-01-01-000001`.
9396

9497
include::../follow-request-body.asciidoc[]
9598

‎docs/reference/ccr/apis/follow/put-follow.asciidoc‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,26 @@ referenced leader index. When this API returns, the follower index exists, and
7676
(Required, string) The <<remote-clusters,remote cluster>> containing
7777
the leader index.
7878

79+
[[ccr-put-follow-request-body-data_stream_name]]`data_stream_name`::
80+
(Optional, string) If the leader index is part of a <<data-streams,data stream>>, the name to
81+
which the local data stream for the followed index should be renamed. For example, A request like:
82+
83+
[source,console]
84+
--------------------------------------------------
85+
PUT /.ds-logs-mysql-default_copy-2022-01-01-000001/_ccr/follow
86+
{
87+
"remote_cluster" : "remote_cluster",
88+
"leader_index" : ".ds-logs-mysql-default-2022-01-01-000001",
89+
"data_stream_name": "logs-mysql-default_copy"
90+
}
91+
--------------------------------------------------
92+
// TEST[skip:no setup]
93+
94+
Replicates the leader index `.ds-logs-mysql-default-2022-01-01-000001` into the follower index
95+
`.ds-logs-mysql-default_copy-2022-01-01-000001` and will do so using the data stream
96+
`logs-mysql-default_copy`, as opposed to the original leader data stream name of
97+
`logs-mysql-default`.
98+
7999
include::../follow-request-body.asciidoc[]
80100

81101
[[ccr-put-follow-examples]]

‎x-pack/plugin/ccr/qa/multi-cluster/src/test/java/org/elasticsearch/xpack/ccr/AutoFollowIT.java‎

Lines changed: 122 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -236,7 +236,7 @@ public void testDataStreams() throws Exception {
236236
int initialNumberOfSuccessfulFollowedIndices = getNumberOfSuccessfulFollowedIndices();
237237
try {
238238
// Create auto follow pattern
239-
createAutoFollowPattern(client(), autoFollowPatternName, "logs-mysql-*", "leader_cluster");
239+
createAutoFollowPattern(client(), autoFollowPatternName, "logs-mysql-*", "leader_cluster", null);
240240

241241
// Create data stream and ensure that is is auto followed
242242
try (RestClient leaderClient = buildLeaderClient()) {
@@ -320,6 +320,121 @@ public void testDataStreams() throws Exception {
320320
}
321321
}
322322

323+
public void testDataStreamsRenameFollowDataStream() throws Exception {
324+
if ("follow".equals(targetCluster) == false) {
325+
return;
326+
}
327+
328+
final int numDocs = 64;
329+
final String dataStreamName = "logs-mysql-error";
330+
final String dataStreamNameFollower = "logs-mysql-error_copy";
331+
final String autoFollowPatternName = getTestName().toLowerCase(Locale.ROOT);
332+
333+
int initialNumberOfSuccessfulFollowedIndices = getNumberOfSuccessfulFollowedIndices();
334+
try {
335+
// Create auto follow pattern
336+
createAutoFollowPattern(client(), autoFollowPatternName, "logs-mysql-*", "leader_cluster", "{{leader_index}}_copy");
337+
338+
// Create data stream and ensure that is is auto followed
339+
try (RestClient leaderClient = buildLeaderClient()) {
340+
for (int i = 0; i < numDocs; i++) {
341+
Request indexRequest = new Request("POST", "/" + dataStreamName + "/_doc");
342+
indexRequest.addParameter("refresh", "true");
343+
indexRequest.setJsonEntity("{\"@timestamp\": \"" + DATE_FORMAT.format(new Date()) + "\",\"message\":\"abc\"}");
344+
assertOK(leaderClient.performRequest(indexRequest));
345+
}
346+
verifyDataStream(leaderClient, dataStreamName, backingIndexName(dataStreamName, 1));
347+
verifyDocuments(leaderClient, dataStreamName, numDocs);
348+
}
349+
logger.info(
350+
"--> checking {} with index {} has been auto followed to {} with backing index {}",
351+
dataStreamName,
352+
backingIndexName(dataStreamName, 1),
353+
dataStreamNameFollower,
354+
backingIndexName(dataStreamNameFollower, 1)
355+
);
356+
assertBusy(() -> {
357+
assertThat(getNumberOfSuccessfulFollowedIndices(), equalTo(initialNumberOfSuccessfulFollowedIndices + 1));
358+
verifyDataStream(client(), dataStreamNameFollower, backingIndexName(dataStreamNameFollower, 1));
359+
ensureYellow(dataStreamNameFollower);
360+
verifyDocuments(client(), dataStreamNameFollower, numDocs);
361+
});
362+
363+
// First rollover and ensure second backing index is replicated:
364+
logger.info("--> rolling over");
365+
try (RestClient leaderClient = buildLeaderClient()) {
366+
Request rolloverRequest = new Request("POST", "/" + dataStreamName + "/_rollover");
367+
assertOK(leaderClient.performRequest(rolloverRequest));
368+
verifyDataStream(leaderClient, dataStreamName, backingIndexName(dataStreamName, 1), backingIndexName(dataStreamName, 2));
369+
370+
Request indexRequest = new Request("POST", "/" + dataStreamName + "/_doc");
371+
indexRequest.addParameter("refresh", "true");
372+
indexRequest.setJsonEntity("{\"@timestamp\": \"" + DATE_FORMAT.format(new Date()) + "\",\"message\":\"abc\"}");
373+
assertOK(leaderClient.performRequest(indexRequest));
374+
verifyDocuments(leaderClient, dataStreamName, numDocs + 1);
375+
}
376+
assertBusy(() -> {
377+
assertThat(getNumberOfSuccessfulFollowedIndices(), equalTo(initialNumberOfSuccessfulFollowedIndices + 2));
378+
verifyDataStream(
379+
client(),
380+
dataStreamNameFollower,
381+
backingIndexName(dataStreamNameFollower, 1),
382+
backingIndexName(dataStreamNameFollower, 2)
383+
);
384+
ensureYellow(dataStreamNameFollower);
385+
verifyDocuments(client(), dataStreamNameFollower, numDocs + 1);
386+
});
387+
388+
// Second rollover and ensure third backing index is replicated:
389+
logger.info("--> rolling over");
390+
try (RestClient leaderClient = buildLeaderClient()) {
391+
Request rolloverRequest = new Request("POST", "/" + dataStreamName + "/_rollover");
392+
assertOK(leaderClient.performRequest(rolloverRequest));
393+
verifyDataStream(
394+
leaderClient,
395+
dataStreamName,
396+
backingIndexName(dataStreamName, 1),
397+
backingIndexName(dataStreamName, 2),
398+
backingIndexName(dataStreamName, 3)
399+
);
400+
401+
Request indexRequest = new Request("POST", "/" + dataStreamName + "/_doc");
402+
indexRequest.addParameter("refresh", "true");
403+
indexRequest.setJsonEntity("{\"@timestamp\": \"" + DATE_FORMAT.format(new Date()) + "\",\"message\":\"abc\"}");
404+
assertOK(leaderClient.performRequest(indexRequest));
405+
verifyDocuments(leaderClient, dataStreamName, numDocs + 2);
406+
}
407+
assertBusy(() -> {
408+
assertThat(getNumberOfSuccessfulFollowedIndices(), equalTo(initialNumberOfSuccessfulFollowedIndices + 3));
409+
verifyDataStream(
410+
client(),
411+
dataStreamNameFollower,
412+
backingIndexName(dataStreamNameFollower, 1),
413+
backingIndexName(dataStreamNameFollower, 2),
414+
backingIndexName(dataStreamNameFollower, 3)
415+
);
416+
ensureYellow(dataStreamNameFollower);
417+
verifyDocuments(client(), dataStreamNameFollower, numDocs + 2);
418+
});
419+
420+
} finally {
421+
cleanUpFollower(
422+
List.of(
423+
backingIndexName(dataStreamNameFollower, 1),
424+
backingIndexName(dataStreamNameFollower, 2),
425+
backingIndexName(dataStreamNameFollower, 3)
426+
),
427+
List.of(dataStreamNameFollower),
428+
List.of(autoFollowPatternName)
429+
);
430+
cleanUpLeader(
431+
List.of(backingIndexName(dataStreamName, 1), backingIndexName(dataStreamName, 2), backingIndexName(dataStreamName, 3)),
432+
List.of(dataStreamName),
433+
List.of()
434+
);
435+
}
436+
}
437+
323438
public void testDataStreams_autoFollowAfterDataStreamCreated() throws Exception {
324439
if ("follow".equals(targetCluster) == false) {
325440
return;
@@ -353,7 +468,7 @@ public void testDataStreams_autoFollowAfterDataStreamCreated() throws Exception
353468
}
354469

355470
// Create auto follow pattern
356-
createAutoFollowPattern(client(), autoFollowPatternName, dataStreamName + "*", "leader_cluster");
471+
createAutoFollowPattern(client(), autoFollowPatternName, dataStreamName + "*", "leader_cluster", null);
357472

358473
// Rollover and ensure only second backing index is replicated:
359474
try (RestClient leaderClient = buildLeaderClient()) {
@@ -410,7 +525,7 @@ public void testRolloverDataStreamInFollowClusterForbidden() throws Exception {
410525
List<String> backingIndexNames = null;
411526
try {
412527
// Create auto follow pattern
413-
createAutoFollowPattern(client(), autoFollowPatternName, "logs-tomcat-*", "leader_cluster");
528+
createAutoFollowPattern(client(), autoFollowPatternName, "logs-tomcat-*", "leader_cluster", null);
414529

415530
// Create data stream and ensure that is is auto followed
416531
try (var leaderClient = buildLeaderClient()) {
@@ -531,7 +646,7 @@ public void testRolloverAliasInFollowClusterForbidden() throws Exception {
531646
int initialNumberOfSuccessfulFollowedIndices = getNumberOfSuccessfulFollowedIndices();
532647
try {
533648
// Create auto follow pattern
534-
createAutoFollowPattern(client(), "test_pattern", "log-*", "leader_cluster");
649+
createAutoFollowPattern(client(), "test_pattern", "log-*", "leader_cluster", null);
535650

536651
// Create leader index and write alias:
537652
try (var leaderClient = buildLeaderClient()) {
@@ -618,7 +733,7 @@ public void testDataStreamsBiDirectionalReplication() throws Exception {
618733

619734
try {
620735
// Create auto follow pattern in follow cluster
621-
createAutoFollowPattern(client(), "id1", "logs-*-eu", "leader_cluster");
736+
createAutoFollowPattern(client(), "id1", "logs-*-eu", "leader_cluster", null);
622737

623738
// Create auto follow pattern in leader cluster:
624739
try (var leaderClient = buildLeaderClient()) {
@@ -658,7 +773,7 @@ public void testDataStreamsBiDirectionalReplication() throws Exception {
658773
}
659774
assertOK(leaderClient.performRequest(request));
660775
// Then create the actual auto follow pattern:
661-
createAutoFollowPattern(leaderClient, "id2", "logs-*-na", "follower_cluster");
776+
createAutoFollowPattern(leaderClient, "id2", "logs-*-na", "follower_cluster", null);
662777
}
663778

664779
var numDocs = 128;
@@ -832,7 +947,7 @@ public void testAutoFollowSearchableSnapshotsFails() throws Exception {
832947
final String mountedIndex = testPrefix + "-mounted";
833948

834949
try {
835-
createAutoFollowPattern(client(), autoFollowPattern, testPrefix + "-*", "leader_cluster");
950+
createAutoFollowPattern(client(), autoFollowPattern, testPrefix + "-*", "leader_cluster", null);
836951

837952
// Create a regular index on leader
838953
try (var leaderClient = buildLeaderClient()) {

‎x-pack/plugin/ccr/qa/multi-cluster/src/test/java/org/elasticsearch/xpack/ccr/FollowIndexIT.java‎

Lines changed: 0 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -180,26 +180,6 @@ public void testFollowDataStreamFails() throws Exception {
180180
assertThat(failure.getMessage(), containsString("cannot follow [logs-syslog-prod], because it is a DATA_STREAM"));
181181
}
182182

183-
public void testChangeBackingIndexNameFails() throws Exception {
184-
if ("follow".equals(targetCluster) == false) {
185-
return;
186-
}
187-
188-
final String dataStreamName = "logs-foobar-prod";
189-
try (RestClient leaderClient = buildLeaderClient()) {
190-
Request request = new Request("PUT", "/_data_stream/" + dataStreamName);
191-
assertOK(leaderClient.performRequest(request));
192-
verifyDataStream(leaderClient, dataStreamName, DataStream.getDefaultBackingIndexName("logs-foobar-prod", 1));
193-
}
194-
195-
ResponseException failure = expectThrows(
196-
ResponseException.class,
197-
() -> followIndex(DataStream.getDefaultBackingIndexName("logs-foobar-prod", 1), ".ds-logs-barbaz-prod-000001")
198-
);
199-
assertThat(failure.getResponse().getStatusLine().getStatusCode(), equalTo(400));
200-
assertThat(failure.getMessage(), containsString("a backing index name in the local and remote cluster must remain the same"));
201-
}
202-
203183
public void testFollowSearchableSnapshotsFails() throws Exception {
204184
final String testPrefix = getTestName().toLowerCase(Locale.ROOT);
205185

‎x-pack/plugin/ccr/qa/security/src/test/java/org/elasticsearch/xpack/ccr/FollowIndexSecurityIT.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -281,7 +281,7 @@ public void testUnPromoteAndFollowDataStream() throws Exception {
281281

282282
// Setup
283283
{
284-
createAutoFollowPattern(adminClient(), "test_pattern", "logs-eu*", "leader_cluster");
284+
createAutoFollowPattern(adminClient(), "test_pattern", "logs-eu*", "leader_cluster", null);
285285
}
286286
// Create data stream and ensure that it is auto followed
287287
{

‎x-pack/plugin/ccr/qa/src/main/java/org/elasticsearch/xpack/ccr/ESCCRRestTestCase.java‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -335,7 +335,13 @@ protected static List<String> verifyDataStream(final RestClient client, final St
335335
return List.copyOf(actualBackingIndices);
336336
}
337337

338-
protected static void createAutoFollowPattern(RestClient client, String name, String pattern, String remoteCluster) throws IOException {
338+
protected static void createAutoFollowPattern(
339+
RestClient client,
340+
String name,
341+
String pattern,
342+
String remoteCluster,
343+
String followIndexPattern
344+
) throws IOException {
339345
Request request = new Request("PUT", "/_ccr/auto_follow/" + name);
340346
try (XContentBuilder bodyBuilder = JsonXContent.contentBuilder()) {
341347
bodyBuilder.startObject();
@@ -345,6 +351,9 @@ protected static void createAutoFollowPattern(RestClient client, String name, St
345351
bodyBuilder.value(pattern);
346352
}
347353
bodyBuilder.endArray();
354+
if (followIndexPattern != null) {
355+
bodyBuilder.field("follow_index_pattern", followIndexPattern);
356+
}
348357
bodyBuilder.field("remote_cluster", remoteCluster);
349358
}
350359
bodyBuilder.endObject();

0 commit comments

Comments
 (0)