[esArchiver] v6 kibana index support#12854
Closed
spalger wants to merge 15 commits intoelastic:masterfrom
Closed
Conversation
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
When creating an id, we will always prefix the type to either an id which was provided or one we generate. This alows us to simplify all actions for managing documents. Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
Remove entire compatibility layer and updates id generation
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
f732b17 to
3e2383d
Compare
afdcd50 to
95b3063
Compare
95b3063 to
3ab51d6
Compare
Contributor
Author
|
Merged esArchive updates into #12794, since |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requires #12794
When we remove support for v5 kibana indices we will need to update our es archives to use v6 format. To do that we add a
convertToV6Streamto the esArchiver that converts doc and index records to v6 format on the fly. This enabled:loadcommand, which write the data into elasticsearch in v6 format which powers...reindexcommand which will read data from elasticsearch in v5 format and put it back into elasticsearch in v6 formatrebuild-allcommand to read the archives from disk in v5 format and write them back to disk in v6 formatAs always, use
node scripts/es_archiver --helpfor CLI syntax. For the impatient, the following will read the .kibana index (or any indices specified afterreindex) to a tmp directory on disk, then delete the indices, and recreate them by first converting the docs into v6 formatThe existing archives were also updated to use v6 format using the
rebuild-allcommand.