- 
                Notifications
    You must be signed in to change notification settings 
- Fork 25.6k
ILM migrate data between tiers #61377
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
          
     Merged
      
      
            andreidan
  merged 42 commits into
  elastic:master
from
andreidan:ilm-migrate-data-between-tiers
  
      
      
   
  Sep 17, 2020 
      
    
  
     Merged
                    Changes from 35 commits
      Commits
    
    
            Show all changes
          
          
            42 commits
          
        
        Select commit
          Hold shift + click to select a range
      
      f4a37a9
              
                Enable the _tier attribute as a node filter
              
              
                andreidan 024a0de
              
                ILM: inject a migrate step to migrate data between data tiers
              
              
                andreidan ca9800f
              
                Merge branch 'master' into ilm-migrate-data-between-tiers
              
              
                andreidan 07dc8de
              
                Add a DataTierMigrationRoutedStep and cleanup
              
              
                andreidan 42900b2
              
                Fix tests
              
              
                andreidan 5d6729b
              
                Fix test
              
              
                andreidan 28ccee9
              
                Remove invalid tier name setting validation from ILM step
              
              
                andreidan 45d2eb3
              
                Add test for DataTierMigrationRoutedStep
              
              
                andreidan 6c05591
              
                Disable data tier migration in the full policy we use for timeseries ITS
              
              
                andreidan 783061d
              
                Remove unused import
              
              
                andreidan ab3ac33
              
                Revert "Disable data tier migration in the full policy we use for tim…
              
              
                andreidan 6b74d7c
              
                Add MigrateActionTest
              
              
                andreidan 61ed9eb
              
                Merge branch 'master' into ilm-migrate-data-between-tiers
              
              
                elasticmachine 6f83f91
              
                Don't randomly pick a disabled migrate action as it has no steps
              
              
                andreidan 4bef9b2
              
                Add ILM tier migratino IT
              
              
                andreidan 5024595
              
                Use 0 replicas in docs test as `check-migration` waits for replicas t…
              
              
                andreidan 41dc480
              
                Fix ILMHistoryTests
              
              
                andreidan d9b6390
              
                Merge branch 'master' into ilm-migrate-data-between-tiers
              
              
                elasticmachine 8e9a4f3
              
                Disable migrate action in test that waits for shard allocations
              
              
                andreidan 996c747
              
                Add ILM client MigrateAction
              
              
                andreidan 83c31b6
              
                Remove unused import
              
              
                andreidan c1746d4
              
                Fix HLRC tests
              
              
                andreidan fe1fb8f
              
                Add tests for conflicting auto allocate to hot nodes and MigrateActio…
              
              
                andreidan e435a2f
              
                Use valid tier name in test
              
              
                andreidan 656afe5
              
                Implement hashcode
              
              
                andreidan a24b0fc
              
                Merge branch 'master' into ilm-migrate-data-between-tiers
              
              
                andreidan 8d806e2
              
                Make method static
              
              
                andreidan 2133715
              
                Check for "data" role in `check-migration` step
              
              
                andreidan 991fdc0
              
                Document AllocationInfo and rename `actual_replicas` to `number_of_re…
              
              
                andreidan be65929
              
                Disallow the migrate action in the hot phase
              
              
                andreidan 39b518e
              
                Remove rogue ;
              
              
                andreidan f3a64d3
              
                Fix HLRC tests
              
              
                andreidan d119797
              
                Merge branch 'master' into ilm-migrate-data-between-tiers
              
              
                elasticmachine 8f855c5
              
                Fix ILMHistoryTests
              
              
                andreidan 0636d0e
              
                Merge branch 'master' into ilm-migrate-data-between-tiers
              
              
                elasticmachine 500a275
              
                Merge branch 'master' into ilm-migrate-data-between-tiers
              
              
                elasticmachine 9b90555
              
                Remove frozen phase from HLRC
              
              
                andreidan eddd223
              
                Execute the allocate action before migrate
              
              
                andreidan d733d72
              
                Adjust test to wait until ILM completes in the cold phase
              
              
                andreidan 4da7379
              
                Log and exception messages
              
              
                andreidan be17159
              
                Merge branch 'master' into ilm-migrate-data-between-tiers
              
              
                andreidan b837718
              
                Drop equals and hashcode (in favour of the ones in Step)
              
              
                andreidan File filter
Filter by extension
Conversations
          Failed to load comments.   
        
        
          
      Loading
        
  Jump to
        
          Jump to file
        
      
      
          Failed to load files.   
        
        
          
      Loading
        
  Diff view
Diff view
There are no files selected for viewing
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
        
          
          
            90 changes: 90 additions & 0 deletions
          
          90 
        
  client/rest-high-level/src/main/java/org/elasticsearch/client/ilm/MigrateAction.java
  
  
      
      
   
        
      
      
    
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,90 @@ | ||
| /* | ||
| * Licensed to Elasticsearch under one or more contributor | ||
| * license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright | ||
| * ownership. Elasticsearch licenses this file to you under | ||
| * the Apache License, Version 2.0 (the "License"); you may | ||
| * not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, | ||
| * software distributed under the License is distributed on an | ||
| * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| * KIND, either express or implied. See the License for the | ||
| * specific language governing permissions and limitations | ||
| * under the License. | ||
| */ | ||
| package org.elasticsearch.client.ilm; | ||
|  | ||
| import org.elasticsearch.common.ParseField; | ||
| import org.elasticsearch.common.Strings; | ||
| import org.elasticsearch.common.xcontent.ConstructingObjectParser; | ||
| import org.elasticsearch.common.xcontent.ToXContentObject; | ||
| import org.elasticsearch.common.xcontent.XContentBuilder; | ||
| import org.elasticsearch.common.xcontent.XContentParser; | ||
|  | ||
| import java.io.IOException; | ||
| import java.util.Objects; | ||
|  | ||
| public class MigrateAction implements LifecycleAction, ToXContentObject { | ||
| public static final String NAME = "migrate"; | ||
|  | ||
| public static final ParseField ENABLED_FIELD = new ParseField("enabled"); | ||
|  | ||
| private static final ConstructingObjectParser<MigrateAction, Void> PARSER = new ConstructingObjectParser<>(NAME, | ||
| a -> new MigrateAction(a[0] == null ? true : (boolean) a[0])); | ||
|  | ||
| static { | ||
| PARSER.declareBoolean(ConstructingObjectParser.optionalConstructorArg(), ENABLED_FIELD); | ||
| } | ||
|  | ||
| public static MigrateAction parse(XContentParser parser) { | ||
| return PARSER.apply(parser, null); | ||
| } | ||
|  | ||
| private final boolean enabled; | ||
|  | ||
| public MigrateAction() { | ||
| this(true); | ||
| } | ||
|  | ||
| public MigrateAction(boolean enabled) { | ||
| this.enabled = enabled; | ||
| } | ||
|  | ||
| @Override | ||
| public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException { | ||
| builder.startObject(); | ||
| builder.field(ENABLED_FIELD.getPreferredName(), enabled); | ||
| builder.endObject(); | ||
| return builder; | ||
| } | ||
|  | ||
| @Override | ||
| public String getName() { | ||
| return NAME; | ||
| } | ||
|  | ||
| @Override | ||
| public int hashCode() { | ||
| return Objects.hashCode(enabled); | ||
| } | ||
|  | ||
| @Override | ||
| public boolean equals(Object obj) { | ||
| if (obj == null) { | ||
| return false; | ||
| } | ||
| if (obj.getClass() != getClass()) { | ||
| return false; | ||
| } | ||
| return true; | ||
| } | ||
|  | ||
| @Override | ||
| public String toString() { | ||
| return Strings.toString(this); | ||
| } | ||
| } | 
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
        
          
          
            46 changes: 46 additions & 0 deletions
          
          46 
        
  client/rest-high-level/src/test/java/org/elasticsearch/client/ilm/MigrateActionTests.java
  
  
      
      
   
        
      
      
    
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| /* | ||
| * Licensed to Elasticsearch under one or more contributor | ||
| * license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright | ||
| * ownership. Elasticsearch licenses this file to you under | ||
| * the Apache License, Version 2.0 (the "License"); you may | ||
| * not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, | ||
| * software distributed under the License is distributed on an | ||
| * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| * KIND, either express or implied. See the License for the | ||
| * specific language governing permissions and limitations | ||
| * under the License. | ||
| */ | ||
| package org.elasticsearch.client.ilm; | ||
|  | ||
| import org.elasticsearch.common.xcontent.XContentParser; | ||
| import org.elasticsearch.test.AbstractXContentTestCase; | ||
|  | ||
| import java.io.IOException; | ||
|  | ||
| public class MigrateActionTests extends AbstractXContentTestCase<MigrateAction> { | ||
|  | ||
| @Override | ||
| protected MigrateAction doParseInstance(XContentParser parser) throws IOException { | ||
| return MigrateAction.parse(parser); | ||
| } | ||
|  | ||
| @Override | ||
| protected MigrateAction createTestInstance() { | ||
| return randomInstance(); | ||
| } | ||
|  | ||
| static MigrateAction randomInstance() { | ||
| return new MigrateAction(randomBoolean()); | ||
| } | ||
|  | ||
| @Override | ||
| protected boolean supportsUnknownFields() { | ||
| return false; | ||
| } | ||
| } | 
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
      
      Oops, something went wrong.
        
    
  
  Add this suggestion to a batch that can be applied as a single commit.
  This suggestion is invalid because no changes were made to the code.
  Suggestions cannot be applied while the pull request is closed.
  Suggestions cannot be applied while viewing a subset of changes.
  Only one suggestion per line can be applied in a batch.
  Add this suggestion to a batch that can be applied as a single commit.
  Applying suggestions on deleted lines is not supported.
  You must change the existing code in this line in order to create a valid suggestion.
  Outdated suggestions cannot be applied.
  This suggestion has been applied or marked resolved.
  Suggestions cannot be applied from pending reviews.
  Suggestions cannot be applied on multi-line comments.
  Suggestions cannot be applied while the pull request is queued to merge.
  Suggestion cannot be applied right now. Please check back later.
  
    
  
    
Uh oh!
There was an error while loading. Please reload this page.