Skip to content

[CCR] Tell user when multiple auto-follow patterns try to replicate the same data#27783

Merged
sebelga merged 5 commits intoelastic:masterfrom
sebelga:fix/auto_follow_pattern-when-multiple-auto-follow-patterns-try-to-replicate-the-same-data
Dec 28, 2018
Merged

[CCR] Tell user when multiple auto-follow patterns try to replicate the same data#27783
sebelga merged 5 commits intoelastic:masterfrom
sebelga:fix/auto_follow_pattern-when-multiple-auto-follow-patterns-try-to-replicate-the-same-data

Conversation

@sebelga
Copy link
Copy Markdown
Contributor

@sebelga sebelga commented Dec 26, 2018

In order to fix a bug where the UI silently fail when 2 auto-follow patterns try to replicate the same index on the leader (see issue #27428) we need to fetch the CCR stats and display the possible errors for an auto-follow pattern.

This PR add 3 API endpoints for the stats

  • GET /cross_cluster_replication/stats (all the stats coming from ES)
  • GET /cross_cluster_replication/stats/auto-follow (only the auto-follow pattern stats)

Test

In order to reproduce the error:

  • Create 2 auto-follow pattern with a similar leader index pattern (e.g. kiba* and kibana*
  • Create an index on the remote cluster that matches both patterns. e.g:
PUT /kibana_sample

# body
{
  "settings": {
    "number_of_shards": 1,
    "soft_deletes.enabled": true
  }
}

screen shot 2018-12-27 at 12 44 51

Fixes #27428

@sebelga sebelga requested a review from jen-huang December 26, 2018 15:55
@elasticmachine
Copy link
Copy Markdown
Contributor

💔 Build Failed

@sebelga sebelga added v7.0.0 v6.6.0 v6.7.0 Feature:CCR and Remote Clusters Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more t// labels Dec 26, 2018
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/es-ui

@sebelga
Copy link
Copy Markdown
Contributor Author

sebelga commented Dec 26, 2018

retest

@elasticmachine
Copy link
Copy Markdown
Contributor

💔 Build Failed

@sebelga
Copy link
Copy Markdown
Contributor Author

sebelga commented Dec 27, 2018

@jen-huang I added the UI to fix the bug so you can test the complete flow. cheers!

@elasticmachine
Copy link
Copy Markdown
Contributor

💔 Build Failed

…attern-when-multiple-auto-follow-patterns-try-to-replicate-the-same-data
@elasticmachine
Copy link
Copy Markdown
Contributor

💚 Build Succeeded

@sebelga sebelga changed the title [CCR] Add API to fetch ccr stats [CCR] Tell user when multiple auto-follow patterns try to replicate the same data Dec 27, 2018
Copy link
Copy Markdown
Contributor

@jen-huang jen-huang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice, tested locally and code LGTM!

A timestamp field was added in ES for 6.7: elastic/elasticsearch#36947

We should conditionally show this information if it exists - but that can be done in a follow up PR since it's not needed for 6.6!

@sebelga
Copy link
Copy Markdown
Contributor Author

sebelga commented Dec 28, 2018

Great thanks for the update on ES timestamp. We should ask them to ping us in the ticket issue when merging the PR for more visibility I think.

@sebelga sebelga merged commit 93e4f12 into elastic:master Dec 28, 2018
@sebelga sebelga deleted the fix/auto_follow_pattern-when-multiple-auto-follow-patterns-try-to-replicate-the-same-data branch December 28, 2018 06:50
sebelga added a commit to sebelga/kibana that referenced this pull request Dec 28, 2018
…he same data (elastic#27783)

* Fix small tipo

* [CCR] Add API to fetch ccr stats

* Add deserialization for recentAutoFollowErrors & autoFollowedClusters

* Add UI to display recent auto-follow pattern errors
sebelga added a commit to sebelga/kibana that referenced this pull request Dec 28, 2018
…he same data (elastic#27783)

* Fix small tipo

* [CCR] Add API to fetch ccr stats

* Add deserialization for recentAutoFollowErrors & autoFollowedClusters

* Add UI to display recent auto-follow pattern errors
sebelga added a commit that referenced this pull request Dec 28, 2018
…he same data (#27783) (#27833)

* Fix small tipo

* [CCR] Add API to fetch ccr stats

* Add deserialization for recentAutoFollowErrors & autoFollowedClusters

* Add UI to display recent auto-follow pattern errors
sebelga added a commit that referenced this pull request Dec 28, 2018
…he same data (#27783) (#27834)

* Fix small tipo

* [CCR] Add API to fetch ccr stats

* Add deserialization for recentAutoFollowErrors & autoFollowedClusters

* Add UI to display recent auto-follow pattern errors
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Feature:CCR and Remote Clusters Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more t// v6.6.0 v6.7.0 v7.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CCR] Tell user when multiple auto-follow patterns try to replicate the same data

3 participants