Skip to content

Add migrate to data tiers API#74264

Merged
andreidan merged 10 commits intoelastic:masterfrom
andreidan:migrate-to-dt-api
Jun 28, 2021
Merged

Add migrate to data tiers API#74264
andreidan merged 10 commits intoelastic:masterfrom
andreidan:migrate-to-dt-api

Conversation

@andreidan
Copy link
Contributor

@andreidan andreidan commented Jun 17, 2021

This adds the API to expose the service for migrating the elasticsearch
abstractions (indices, ILM policies and an optional legacy template to
delete) to data tiers routing allocation (away from custom node attributes).
The details for how the migration is working are specified in #73689

The API is

POST /_ilm/migrate_to_data_tiers
{
  "legacy_template_to_delete": "template_name",
  "node_attribute": "data"
}

The response will contain the name of the deleted legacy index template
(if any was deleted), a list of ILM policies and a list of indices names containing
the policies and indices that were migrated.

{
  "removed_legacy_template":"catch-all-temaplte",
  "migrated_ilm_policies":["policy-one","policy-two"],
  "migrated_indices":["logs-000001","indexwithdatawarmrouting",".ds-.slm-history-5-2021.04.22-000002"]
}

Relates to #73154

This adds the API to expose the service for migrating the elasticsearch
abstractions (indices, ILM policies and an optional legacy template to
delete) to data tiers routing allocation (away from custom node attributes)
@andreidan andreidan added :Data Management/Indices APIs DO NOT USE. Use ":Distributed/Indices APIs" or ":StorageEngine/Templates" instead. :Data Management/ILM+SLM DO NOT USE. Use ":StorageEngine/ILM" or ":Distributed Coordination/SLM" instead. v8.0.0 v7.14.0 labels Jun 17, 2021
@andreidan andreidan requested a review from dakrone June 17, 2021 19:55
@elasticmachine elasticmachine added the Team:Data Management (obsolete) DO NOT USE. This team no longer exists. label Jun 17, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features (Team:Core/Features)

@andreidan
Copy link
Contributor Author

@elasticmachine update branch

@andreidan
Copy link
Contributor Author

@elasticmachine run elasticsearch-ci/part-2

@andreidan andreidan added the >docs General docs changes label Jun 18, 2021
@elasticmachine elasticmachine added the Team:Docs Meta label for docs team label Jun 18, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-docs (Team:Docs)

@sethmlarson sethmlarson added the Team:Clients Meta label for clients team label Jun 21, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/clients-team (Team:Clients)

@cjcenizal cjcenizal added the Team:Deployment Management Meta label for Management Experience - Deployment Management team label Jun 21, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/kibana-stack-management (Team:Stack Management)

@sethmlarson
Copy link
Contributor

Would this API be better suited in a namespace? Since it appears to be documented nearby ILM APIs perhaps the ilm namespace is a good choice (despite of course making changes to indices/templates as well)?

@jethr0null
Copy link

@dakrone @andreidan @sajjadwahmed could you please help us to understand the desired UX/user flows for this change (especially as it relates to any requested UI updates)?

Copy link
Member

@dakrone dakrone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM, thanks for working on it Andrei. I do think this should be nested within the /_ilm/ namespace, so POST /_ilm/migrate_to_data_tiers, as I think it fits best there rather than a top-level API. What do you think?

@dakrone
Copy link
Member

dakrone commented Jun 24, 2021

could you please help us to understand the desired UX/user flows for this change (especially as it relates to any requested UI updates)?

I don't think we currently have plans to expose this as part of the UI (please correct me if I'm mistaken @andreidan), but this would generally be something invoked as a one-off by an administrator or at the behest of support to help migrate without having to manually do the migration.

@andreidan
Copy link
Contributor Author

@dakrone thanks for the review. Nested API under the /_ilm/ namespace. The API now is _ilm/migrate_to_data_tiers

@andreidan andreidan merged commit 636aa7c into elastic:master Jun 28, 2021
andreidan added a commit to andreidan/elasticsearch that referenced this pull request Jun 28, 2021
This adds the _ilm/migrate_to_data_tiers API to expose the service for
migrating the elasticsearch abstractions (indices, ILM policies and an
optional legacy template to delete) to data tiers routing allocation
(away from custom node attributes)

(cherry picked from commit 636aa7c)
Signed-off-by: Andrei Dan <andrei.dan@elastic.co>
andreidan added a commit that referenced this pull request Jun 28, 2021
This adds the _ilm/migrate_to_data_tiers API to expose the service for
migrating the elasticsearch abstractions (indices, ILM policies and an
optional legacy template to delete) to data tiers routing allocation
(away from custom node attributes)

(cherry picked from commit 636aa7c)
Signed-off-by: Andrei Dan <andrei.dan@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Data Management/ILM+SLM DO NOT USE. Use ":StorageEngine/ILM" or ":Distributed Coordination/SLM" instead. :Data Management/Indices APIs DO NOT USE. Use ":Distributed/Indices APIs" or ":StorageEngine/Templates" instead. >docs General docs changes release highlight Team:Clients Meta label for clients team Team:Data Management (obsolete) DO NOT USE. This team no longer exists. Team:Deployment Management Meta label for Management Experience - Deployment Management team Team:Docs Meta label for docs team v7.14.0 v8.0.0-alpha1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants