[Obs AI Assistant] Remove index write blocks#220362
Merged
sorenlouv merged 3 commits intoelastic:mainfrom May 8, 2025
Merged
Conversation
sorenlouv
added
backport:version
sorenlouv
added
release_note:enhancement
release_note:fix
and removed
release_note:enhancement
labels
Contributor
|
Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant) |
viduni94
added
the
ci:project-deploy-observability
Contributor
🤖 GitHub commentsExpand to view the GitHub comments
Just comment with:
|
Contributor
|
Added label to deploy serverless project and rebasing to do a MKI run |
viduni94
approved these changes
May 7, 2025
..._agnostic/apis/observability/ai_assistant/knowledge_base/knowledge_base_warmup_model.spec.ts
Outdated
Show resolved
Hide resolved
…lity/ai_assistant/knowledge_base/knowledge_base_warmup_model.spec.ts Co-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>
Contributor
💚 Build Succeeded
Metrics [docs]
History
|
Contributor
|
Starting backport for target branches: 8.19 https://github.com/elastic/kibana/actions/runs/14902396771 |
Contributor
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
Contributor
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#8250[❌] x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts: 99/100 tests passed. |
Member
Author
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
sorenlouv
added a commit
to sorenlouv/kibana
that referenced
this pull request
May 8, 2025
Closes elastic#220248 - This PR removes the write index blocks that were previously added when starting a reindex operation. The index blocks were added to prevent writes to the source index, while it was being re-indexed to a target index. - Re-enables tests to run on MKI Why index blocks should be removed: - Immediately after creating the target index, the KB write index alias is pointed over to the new target index. This means that it is possible to ingest documents to the new target index, while the re-index is in progress. Thus, we no longer need to block writes - Index blocks must be cleaned up. If a node crashes after a re-index operation has been started, the reindex operation itself will continue to run because it is handled by Elasticsearch with all the guarantees it provides. However, the index block will not be removed again, causing a permanent dead lock until customers remove the index block themselves. Customers should not end up in this situation. - Index blocks are causing tests to fail, when the index block is not properly cleaned up afterwards: elastic#220248 --------- Co-authored-by: Viduni Wickramarachchi <viduni.wickramarachchi@elastic.co> Co-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com> (cherry picked from commit 3c95148) # Conflicts: # x-pack/test/api_integration/deployment_agnostic/apis/observability/ai_assistant/knowledge_base/knowledge_base_basic_operations.spec.ts # x-pack/test/api_integration/deployment_agnostic/apis/observability/ai_assistant/knowledge_base/knowledge_base_change_model_from_elser_to_e5.spec.ts # x-pack/test/api_integration/deployment_agnostic/apis/observability/ai_assistant/knowledge_base/knowledge_base_status.spec.ts
sorenlouv
added a commit
that referenced
this pull request
May 8, 2025
# Backport This will backport the following commits from `main` to `8.19`: - [[Obs AI Assistant] Remove index write blocks (#220362)](#220362) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Søren Louv-Jansen","email":"soren.louv@elastic.co"},"sourceCommit":{"committedDate":"2025-05-08T08:42:54Z","message":"[Obs AI Assistant] Remove index write blocks (#220362)\n\nCloses https://github.com/elastic/kibana/issues/220248\n\n- This PR removes the write index blocks that were previously added when\nstarting a reindex operation. The index blocks were added to prevent\nwrites to the source index, while it was being re-indexed to a target\nindex.\n- Re-enables tests to run on MKI\n\nWhy index blocks should be removed:\n\n- Immediately after creating the target index, the KB write index alias\nis pointed over to the new target index. This means that it is possible\nto ingest documents to the new target index, while the re-index is in\nprogress. Thus, we no longer need to block writes\n- Index blocks must be cleaned up. If a node crashes after a re-index\noperation has been started, the reindex operation itself will continue\nto run because it is handled by Elasticsearch with all the guarantees it\nprovides. However, the index block will not be removed again, causing a\npermanent dead lock until customers remove the index block themselves.\nCustomers should not end up in this situation.\n- Index blocks are causing tests to fail, when the index block is not\nproperly cleaned up afterwards:\nhttps://github.com//issues/220248\n\n---------\n\nCo-authored-by: Viduni Wickramarachchi <viduni.wickramarachchi@elastic.co>\nCo-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>","sha":"3c9514816d0d796af0429c388578d9593f1f7aee","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","Team:Obs AI Assistant","ci:project-deploy-observability","backport:version","v9.1.0","v8.19.0"],"title":"[Obs AI Assistant] Remove index write blocks","number":220362,"url":"https://github.com/elastic/kibana/pull/220362","mergeCommit":{"message":"[Obs AI Assistant] Remove index write blocks (#220362)\n\nCloses https://github.com/elastic/kibana/issues/220248\n\n- This PR removes the write index blocks that were previously added when\nstarting a reindex operation. The index blocks were added to prevent\nwrites to the source index, while it was being re-indexed to a target\nindex.\n- Re-enables tests to run on MKI\n\nWhy index blocks should be removed:\n\n- Immediately after creating the target index, the KB write index alias\nis pointed over to the new target index. This means that it is possible\nto ingest documents to the new target index, while the re-index is in\nprogress. Thus, we no longer need to block writes\n- Index blocks must be cleaned up. If a node crashes after a re-index\noperation has been started, the reindex operation itself will continue\nto run because it is handled by Elasticsearch with all the guarantees it\nprovides. However, the index block will not be removed again, causing a\npermanent dead lock until customers remove the index block themselves.\nCustomers should not end up in this situation.\n- Index blocks are causing tests to fail, when the index block is not\nproperly cleaned up afterwards:\nhttps://github.com//issues/220248\n\n---------\n\nCo-authored-by: Viduni Wickramarachchi <viduni.wickramarachchi@elastic.co>\nCo-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>","sha":"3c9514816d0d796af0429c388578d9593f1f7aee"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/220362","number":220362,"mergeCommit":{"message":"[Obs AI Assistant] Remove index write blocks (#220362)\n\nCloses https://github.com/elastic/kibana/issues/220248\n\n- This PR removes the write index blocks that were previously added when\nstarting a reindex operation. The index blocks were added to prevent\nwrites to the source index, while it was being re-indexed to a target\nindex.\n- Re-enables tests to run on MKI\n\nWhy index blocks should be removed:\n\n- Immediately after creating the target index, the KB write index alias\nis pointed over to the new target index. This means that it is possible\nto ingest documents to the new target index, while the re-index is in\nprogress. Thus, we no longer need to block writes\n- Index blocks must be cleaned up. If a node crashes after a re-index\noperation has been started, the reindex operation itself will continue\nto run because it is handled by Elasticsearch with all the guarantees it\nprovides. However, the index block will not be removed again, causing a\npermanent dead lock until customers remove the index block themselves.\nCustomers should not end up in this situation.\n- Index blocks are causing tests to fail, when the index block is not\nproperly cleaned up afterwards:\nhttps://github.com//issues/220248\n\n---------\n\nCo-authored-by: Viduni Wickramarachchi <viduni.wickramarachchi@elastic.co>\nCo-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>","sha":"3c9514816d0d796af0429c388578d9593f1f7aee"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT-->
akowalska622
pushed a commit
to akowalska622/kibana
that referenced
this pull request
May 29, 2025
Closes elastic#220248 - This PR removes the write index blocks that were previously added when starting a reindex operation. The index blocks were added to prevent writes to the source index, while it was being re-indexed to a target index. - Re-enables tests to run on MKI Why index blocks should be removed: - Immediately after creating the target index, the KB write index alias is pointed over to the new target index. This means that it is possible to ingest documents to the new target index, while the re-index is in progress. Thus, we no longer need to block writes - Index blocks must be cleaned up. If a node crashes after a re-index operation has been started, the reindex operation itself will continue to run because it is handled by Elasticsearch with all the guarantees it provides. However, the index block will not be removed again, causing a permanent dead lock until customers remove the index block themselves. Customers should not end up in this situation. - Index blocks are causing tests to fail, when the index block is not properly cleaned up afterwards: elastic#220248 --------- Co-authored-by: Viduni Wickramarachchi <viduni.wickramarachchi@elastic.co> Co-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>
qn895
pushed a commit
to qn895/kibana
that referenced
this pull request
Jun 3, 2025
Closes elastic#220248 - This PR removes the write index blocks that were previously added when starting a reindex operation. The index blocks were added to prevent writes to the source index, while it was being re-indexed to a target index. - Re-enables tests to run on MKI Why index blocks should be removed: - Immediately after creating the target index, the KB write index alias is pointed over to the new target index. This means that it is possible to ingest documents to the new target index, while the re-index is in progress. Thus, we no longer need to block writes - Index blocks must be cleaned up. If a node crashes after a re-index operation has been started, the reindex operation itself will continue to run because it is handled by Elasticsearch with all the guarantees it provides. However, the index block will not be removed again, causing a permanent dead lock until customers remove the index block themselves. Customers should not end up in this situation. - Index blocks are causing tests to fail, when the index block is not properly cleaned up afterwards: elastic#220248 --------- Co-authored-by: Viduni Wickramarachchi <viduni.wickramarachchi@elastic.co> Co-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #220248
Why index blocks should be removed: