[Obs AI Assistant] Replace Task Manager usage with Lock Manager#216916
Merged
sorenlouv merged 33 commits intoelastic:mainfrom Apr 8, 2025
Merged
[Obs AI Assistant] Replace Task Manager usage with Lock Manager#216916sorenlouv merged 33 commits intoelastic:mainfrom
sorenlouv merged 33 commits intoelastic:mainfrom
Conversation
sorenlouv
changed the title
Contributor
|
Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant) |
sorenlouv
force-pushed
the
replace-tasks-with-locks
branch
from
a875c54 to
65b8bca
Compare
sorenlouv
added
the
release_note:skip
Contributor
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#8149[✅] x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts: 100/100 tests passed. |
sorenlouv
changed the title
Member
Author
Contributor
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#8156[❌] x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts: 75/100 tests passed. |
Contributor
💚 Build Succeeded
Metrics [docs]Public APIs missing comments
History
|
sorenlouv
added a commit
that referenced
this pull request
Apr 8, 2025
Related: #216916 The flaky test runner is sometimes throwing this error: ``` └-: withLock API -- | └-> "before all" hook: beforeTestSuite.trigger in "withLock API" | └-> "before all" hook in "withLock API" | └- ✖ fail: Stateful Observability - Deployment-agnostic API integration tests observability AI Assistant LockManager withLock API "before all" hook in "withLock API" | │ ResponseError: {"took":1,"timed_out":false,"total":1,"deleted":0,"batches":1,"version_conflicts":1,"noops":0,"retries":{"bulk":0,"search":0},"throttled_millis":0,"requests_per_second":-1,"throttled_until_millis":0,"failures":[{"index":".kibana_locks-000001","id":"my_lock_with_token_fencing","cause":{"type":"version_conflict_engine_exception","reason":"[my_lock_with_token_fencing]: version conflict, required seqNo [117], primary term[1]. but no document was found","index_uuid":"F_O5sNfQSLqtyPeLexG_Qw","shard":"0","index":".kibana_locks-000001"},"status":409}]} | │ at SniffingTransport._request (node_modules/@elastic/elasticsearch/node_modules/@elastic/transport/src/Transport.ts:605:17) | │ at processTicksAndRejections (node:internal/process/task_queues:95:5) | │ at /opt/buildkite-agent/builds/bk-agent-prod-gcp-1744099176361402458/elastic/kibana-flaky-test-suite-runner/kibana/node_modules/@elastic/elasticsearch/node_modules/@elastic/transport/src/Transport.ts:711:22 | │ at SniffingTransport.request (node_modules/@elastic/elasticsearch/node_modules/@elastic/transport/src/Transport.ts:708:14) | │ at Client.DeleteByQueryApi [as deleteByQuery] (node_modules/@elastic/src/api/api/delete_by_query.ts:143:10) | │ at Context.<anonymous> (distributed_lock_manager.spec.ts:444:9) | │ at Object.apply (wrap_function.js:74:16) ``` The error happens when `release` and `clearAllLocks` simultaneously tries to delete the same document. This PR ensures that `clearAllLocks` will not throw an exception on conflict
Contributor
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#8157[❌] x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts: 73/100 tests passed. |
kibanamachine
added
the
backport missing
Contributor
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
sorenlouv
added
backport:skip
kibanamachine
removed
the
backport missing
sorenlouv
added a commit
that referenced
this pull request
Apr 23, 2025
Closes: #218944 The index template for the Lock Manager was not created, causing index mappings and settings to be incorrect. Root cause: the function responsible for creating the index template (`ensureTemplatesAndIndexCreated`) was never invoked - only during automated testing 🤦 ## Manual testing steps The mappings for the lock manager index (`.kibana_locks-000001`) should match this: ```ts { mappings: { dynamic: false, properties: { token: { type: 'keyword' }, metadata: { enabled: false }, createdAt: { type: 'date' }, expiresAt: { type: 'date' }, }, }, } ``` ### Test 1 In this test we make sure that the LockManager library can recover and fix the mappings if the existing index has invalid mappings #### Delete existing assets if they already exist ``` DELETE .kibana_locks-000001 DELETE _index_template/.kibana_locks-index-template DELETE _component_template/.kibana_locks-component ``` #### Create lock index. This is invalid because it does not have the correct mappings ``` PUT .kibana_locks-000001 ``` (Restart Kibana) #### Check mappings are correct ``` GET .kibana_locks-000001/_mapping ``` ### Test 2 In this test we make sure that out of the box, the LockManager library creates an index with the correct mappings ``` DELETE .kibana_locks-000001 DELETE _index_template/.kibana_locks-index-template DELETE _component_template/.kibana_locks-component ``` (Restart Kibana) #### Check mappings are correct ``` GET .kibana_locks-000001/_mapping ``` Related: #216916 #216397 --------- Co-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>
Member
Author
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
sorenlouv
added a commit
to sorenlouv/kibana
that referenced
this pull request
Apr 30, 2025
Closes: elastic#218944 The index template for the Lock Manager was not created, causing index mappings and settings to be incorrect. Root cause: the function responsible for creating the index template (`ensureTemplatesAndIndexCreated`) was never invoked - only during automated testing 🤦 The mappings for the lock manager index (`.kibana_locks-000001`) should match this: ```ts { mappings: { dynamic: false, properties: { token: { type: 'keyword' }, metadata: { enabled: false }, createdAt: { type: 'date' }, expiresAt: { type: 'date' }, }, }, } ``` In this test we make sure that the LockManager library can recover and fix the mappings if the existing index has invalid mappings ``` DELETE .kibana_locks-000001 DELETE _index_template/.kibana_locks-index-template DELETE _component_template/.kibana_locks-component ``` correct mappings ``` PUT .kibana_locks-000001 ``` (Restart Kibana) ``` GET .kibana_locks-000001/_mapping ``` In this test we make sure that out of the box, the LockManager library creates an index with the correct mappings ``` DELETE .kibana_locks-000001 DELETE _index_template/.kibana_locks-index-template DELETE _component_template/.kibana_locks-component ``` (Restart Kibana) ``` GET .kibana_locks-000001/_mapping ``` Related: elastic#216916 elastic#216397 --------- Co-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com> (cherry picked from commit f684ea4)
sorenlouv
added a commit
to sorenlouv/kibana
that referenced
this pull request
Apr 30, 2025
Closes: elastic#218944 The index template for the Lock Manager was not created, causing index mappings and settings to be incorrect. Root cause: the function responsible for creating the index template (`ensureTemplatesAndIndexCreated`) was never invoked - only during automated testing 🤦 The mappings for the lock manager index (`.kibana_locks-000001`) should match this: ```ts { mappings: { dynamic: false, properties: { token: { type: 'keyword' }, metadata: { enabled: false }, createdAt: { type: 'date' }, expiresAt: { type: 'date' }, }, }, } ``` In this test we make sure that the LockManager library can recover and fix the mappings if the existing index has invalid mappings ``` DELETE .kibana_locks-000001 DELETE _index_template/.kibana_locks-index-template DELETE _component_template/.kibana_locks-component ``` correct mappings ``` PUT .kibana_locks-000001 ``` (Restart Kibana) ``` GET .kibana_locks-000001/_mapping ``` In this test we make sure that out of the box, the LockManager library creates an index with the correct mappings ``` DELETE .kibana_locks-000001 DELETE _index_template/.kibana_locks-index-template DELETE _component_template/.kibana_locks-component ``` (Restart Kibana) ``` GET .kibana_locks-000001/_mapping ``` Related: elastic#216916 elastic#216397 --------- Co-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com> (cherry picked from commit f684ea4)
sorenlouv
added a commit
that referenced
this pull request
May 1, 2025
# Backport This will backport the following commits from `main` to `8.19`: - [[Obs AI Assistant] Skip lock tests in MKI temporarily (#216753)](#216753) - [[Obs AI Assistant] Replace Task Manager usage with Lock Manager (#216916)](#216916) - [[LockManager] Ensure index template are created (#218901)](#218901) - [[LockManager] Expose as package (#219220)](#219220) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Søren Louv-Jansen","email":"soren.louv@elastic.co"},"sourceCommit":{"committedDate":"2025-04-02T09:28:35Z","message":"[Obs AI Assistant] Skip lock tests in MKI temporarily (#216753)\n\nTests added in #216397 are failing\non MKI. Skipping temporarily in the affected environment\n\n### Error\n\n```\n └- ✖ fail: Serverless Observability - Deployment-agnostic API integration tests observability AI Assistant LockManager Basic lock operations acquires the lock when not held\n │ ResponseError: security_exception\n │ \tRoot causes:\n │ \t\tsecurity_exception: action [indices:admin/create] is unauthorized for user [testing-internal] with effective roles [superuser] on restricted indices [.kibana_locks-000001], this action is granted by the index privileges [create_index,manage,all]\n```\n\n### Root cause\n```ts\nconst es = getService('es');\nes.deleteByQuery({ index: '.kibana_locks-000001', query: { match_all: {} }});\n```","sha":"8bcce2e89b6be758d144b9b802eb9d2a09dd62cf","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:skip","v9.1.0","v8.19.0"],"title":"[Obs AI Assistant] Skip lock tests in MKI temporarily","number":216753,"url":"https://github.com/elastic/kibana/pull/216753","mergeCommit":{"message":"[Obs AI Assistant] Skip lock tests in MKI temporarily (#216753)\n\nTests added in #216397 are failing\non MKI. Skipping temporarily in the affected environment\n\n### Error\n\n```\n └- ✖ fail: Serverless Observability - Deployment-agnostic API integration tests observability AI Assistant LockManager Basic lock operations acquires the lock when not held\n │ ResponseError: security_exception\n │ \tRoot causes:\n │ \t\tsecurity_exception: action [indices:admin/create] is unauthorized for user [testing-internal] with effective roles [superuser] on restricted indices [.kibana_locks-000001], this action is granted by the index privileges [create_index,manage,all]\n```\n\n### Root cause\n```ts\nconst es = getService('es');\nes.deleteByQuery({ index: '.kibana_locks-000001', query: { match_all: {} }});\n```","sha":"8bcce2e89b6be758d144b9b802eb9d2a09dd62cf"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/216753","number":216753,"mergeCommit":{"message":"[Obs AI Assistant] Skip lock tests in MKI temporarily (#216753)\n\nTests added in #216397 are failing\non MKI. Skipping temporarily in the affected environment\n\n### Error\n\n```\n └- ✖ fail: Serverless Observability - Deployment-agnostic API integration tests observability AI Assistant LockManager Basic lock operations acquires the lock when not held\n │ ResponseError: security_exception\n │ \tRoot causes:\n │ \t\tsecurity_exception: action [indices:admin/create] is unauthorized for user [testing-internal] with effective roles [superuser] on restricted indices [.kibana_locks-000001], this action is granted by the index privileges [create_index,manage,all]\n```\n\n### Root cause\n```ts\nconst es = getService('es');\nes.deleteByQuery({ index: '.kibana_locks-000001', query: { match_all: {} }});\n```","sha":"8bcce2e89b6be758d144b9b802eb9d2a09dd62cf"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]},{"author":{"name":"Søren Louv-Jansen","email":"soren.louv@elastic.co"},"sourceCommit":{"committedDate":"2025-04-08T08:13:00Z","message":"[Obs AI Assistant] Replace Task Manager usage with Lock Manager (#216916)\n\nThis PR replaces usage of the Kibana Task Manager with the [LockManager\nutility](#216397) that ensures\nstrict mutual exclusion when running operations such as re-indexing or\ndata migrations.\n\n---------\n\nCo-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"624410524f1a07dc72d92759464a3f4a85466c7a","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:skip","Team:Obs AI Assistant","v9.1.0"],"title":"[Obs AI Assistant] Replace Task Manager usage with Lock Manager","number":216916,"url":"https://github.com/elastic/kibana/pull/216916","mergeCommit":{"message":"[Obs AI Assistant] Replace Task Manager usage with Lock Manager (#216916)\n\nThis PR replaces usage of the Kibana Task Manager with the [LockManager\nutility](#216397) that ensures\nstrict mutual exclusion when running operations such as re-indexing or\ndata migrations.\n\n---------\n\nCo-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"624410524f1a07dc72d92759464a3f4a85466c7a"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/216916","number":216916,"mergeCommit":{"message":"[Obs AI Assistant] Replace Task Manager usage with Lock Manager (#216916)\n\nThis PR replaces usage of the Kibana Task Manager with the [LockManager\nutility](#216397) that ensures\nstrict mutual exclusion when running operations such as re-indexing or\ndata migrations.\n\n---------\n\nCo-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"624410524f1a07dc72d92759464a3f4a85466c7a"}}]},{"author":{"name":"Søren Louv-Jansen","email":"soren.louv@elastic.co"},"sourceCommit":{"committedDate":"2025-04-23T23:34:28Z","message":"[LockManager] Ensure index template are created (#218901)\n\nCloses: https://github.com/elastic/kibana/issues/218944\n\nThe index template for the Lock Manager was not created, causing index\nmappings and settings to be incorrect.\nRoot cause: the function responsible for creating the index template\n(`ensureTemplatesAndIndexCreated`) was never invoked - only during\nautomated testing 🤦\n\n## Manual testing steps\n\nThe mappings for the lock manager index (`.kibana_locks-000001`) should\nmatch this:\n```ts\n {\n mappings: {\n dynamic: false,\n properties: {\n token: { type: 'keyword' },\n metadata: { enabled: false },\n createdAt: { type: 'date' },\n expiresAt: { type: 'date' },\n },\n },\n}\n```\n\n### Test 1\n\nIn this test we make sure that the LockManager library can recover and\nfix the mappings if the existing index has invalid mappings\n\n#### Delete existing assets if they already exist\n```\nDELETE .kibana_locks-000001\nDELETE _index_template/.kibana_locks-index-template\nDELETE _component_template/.kibana_locks-component\n```\n\n#### Create lock index. This is invalid because it does not have the\ncorrect mappings\n```\nPUT .kibana_locks-000001\n```\n\n(Restart Kibana)\n\n#### Check mappings are correct\n```\nGET .kibana_locks-000001/_mapping\n```\n\n### Test 2\n\nIn this test we make sure that out of the box, the LockManager library\ncreates an index with the correct mappings\n\n```\nDELETE .kibana_locks-000001\nDELETE _index_template/.kibana_locks-index-template\nDELETE _component_template/.kibana_locks-component\n```\n\n(Restart Kibana)\n\n#### Check mappings are correct\n```\nGET .kibana_locks-000001/_mapping\n```\n\nRelated: https://github.com/elastic/kibana/pull/216916\nhttps://github.com/elastic/kibana/pull/216397\n\n---------\n\nCo-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>","sha":"f684ea4071dfb5e6acc4cb057b46acbf915943d7","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","backport:skip","Team:Obs AI Assistant","ci:project-deploy-observability","v9.1.0"],"title":"[LockManager] Ensure index template are created","number":218901,"url":"https://github.com/elastic/kibana/pull/218901","mergeCommit":{"message":"[LockManager] Ensure index template are created (#218901)\n\nCloses: https://github.com/elastic/kibana/issues/218944\n\nThe index template for the Lock Manager was not created, causing index\nmappings and settings to be incorrect.\nRoot cause: the function responsible for creating the index template\n(`ensureTemplatesAndIndexCreated`) was never invoked - only during\nautomated testing 🤦\n\n## Manual testing steps\n\nThe mappings for the lock manager index (`.kibana_locks-000001`) should\nmatch this:\n```ts\n {\n mappings: {\n dynamic: false,\n properties: {\n token: { type: 'keyword' },\n metadata: { enabled: false },\n createdAt: { type: 'date' },\n expiresAt: { type: 'date' },\n },\n },\n}\n```\n\n### Test 1\n\nIn this test we make sure that the LockManager library can recover and\nfix the mappings if the existing index has invalid mappings\n\n#### Delete existing assets if they already exist\n```\nDELETE .kibana_locks-000001\nDELETE _index_template/.kibana_locks-index-template\nDELETE _component_template/.kibana_locks-component\n```\n\n#### Create lock index. This is invalid because it does not have the\ncorrect mappings\n```\nPUT .kibana_locks-000001\n```\n\n(Restart Kibana)\n\n#### Check mappings are correct\n```\nGET .kibana_locks-000001/_mapping\n```\n\n### Test 2\n\nIn this test we make sure that out of the box, the LockManager library\ncreates an index with the correct mappings\n\n```\nDELETE .kibana_locks-000001\nDELETE _index_template/.kibana_locks-index-template\nDELETE _component_template/.kibana_locks-component\n```\n\n(Restart Kibana)\n\n#### Check mappings are correct\n```\nGET .kibana_locks-000001/_mapping\n```\n\nRelated: https://github.com/elastic/kibana/pull/216916\nhttps://github.com/elastic/kibana/pull/216397\n\n---------\n\nCo-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>","sha":"f684ea4071dfb5e6acc4cb057b46acbf915943d7"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/218901","number":218901,"mergeCommit":{"message":"[LockManager] Ensure index template are created (#218901)\n\nCloses: https://github.com/elastic/kibana/issues/218944\n\nThe index template for the Lock Manager was not created, causing index\nmappings and settings to be incorrect.\nRoot cause: the function responsible for creating the index template\n(`ensureTemplatesAndIndexCreated`) was never invoked - only during\nautomated testing 🤦\n\n## Manual testing steps\n\nThe mappings for the lock manager index (`.kibana_locks-000001`) should\nmatch this:\n```ts\n {\n mappings: {\n dynamic: false,\n properties: {\n token: { type: 'keyword' },\n metadata: { enabled: false },\n createdAt: { type: 'date' },\n expiresAt: { type: 'date' },\n },\n },\n}\n```\n\n### Test 1\n\nIn this test we make sure that the LockManager library can recover and\nfix the mappings if the existing index has invalid mappings\n\n#### Delete existing assets if they already exist\n```\nDELETE .kibana_locks-000001\nDELETE _index_template/.kibana_locks-index-template\nDELETE _component_template/.kibana_locks-component\n```\n\n#### Create lock index. This is invalid because it does not have the\ncorrect mappings\n```\nPUT .kibana_locks-000001\n```\n\n(Restart Kibana)\n\n#### Check mappings are correct\n```\nGET .kibana_locks-000001/_mapping\n```\n\n### Test 2\n\nIn this test we make sure that out of the box, the LockManager library\ncreates an index with the correct mappings\n\n```\nDELETE .kibana_locks-000001\nDELETE _index_template/.kibana_locks-index-template\nDELETE _component_template/.kibana_locks-component\n```\n\n(Restart Kibana)\n\n#### Check mappings are correct\n```\nGET .kibana_locks-000001/_mapping\n```\n\nRelated: https://github.com/elastic/kibana/pull/216916\nhttps://github.com/elastic/kibana/pull/216397\n\n---------\n\nCo-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>","sha":"f684ea4071dfb5e6acc4cb057b46acbf915943d7"}}]},{"author":{"name":"Søren Louv-Jansen","email":"soren.louv@elastic.co"},"sourceCommit":{"committedDate":"2025-04-29T16:42:45Z","message":"[LockManager] Expose as package (#219220)\n\nExpose LockManager as package to make it easier to consume from other\nplugins\n\ncc @nchaulet\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>","sha":"8b8d569986f216185755aa7ac98a2a3bbeb84f76","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Obs AI Assistant","ci:project-deploy-observability","backport:version","v9.1.0","v8.19.0"],"title":"[LockManager] Expose as package","number":219220,"url":"https://github.com/elastic/kibana/pull/219220","mergeCommit":{"message":"[LockManager] Expose as package (#219220)\n\nExpose LockManager as package to make it easier to consume from other\nplugins\n\ncc @nchaulet\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>","sha":"8b8d569986f216185755aa7ac98a2a3bbeb84f76"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/219220","number":219220,"mergeCommit":{"message":"[LockManager] Expose as package (#219220)\n\nExpose LockManager as package to make it easier to consume from other\nplugins\n\ncc @nchaulet\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>","sha":"8b8d569986f216185755aa7ac98a2a3bbeb84f76"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
sorenlouv
added a commit
to sorenlouv/kibana
that referenced
this pull request
May 5, 2025
…tic#216916) This PR replaces usage of the Kibana Task Manager with the [LockManager utility](elastic#216397) that ensures strict mutual exclusion when running operations such as re-indexing or data migrations. --------- Co-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit 6244105)
Member
Author
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
sorenlouv
added a commit
that referenced
this pull request
May 5, 2025
… Assistant] Replace Task Manager usage with Lock Manager (#216916) | update knowledge base installation flow (#214133) | [Obs AI Assistant] Specify embedding model during onboarding for the Knowledge Base (#218448) (#220041) # Backport This will backport the following commits from `main` to `8.19`: - [[Obs AI Assistant] Minor cleanup api tests (#215181)](#215181) - [[Obs AI Assistant] Replace Task Manager usage with Lock Manager (#216916)](#216916) - [update knowledge base installation flow (#214133)](#214133) - [[Obs AI Assistant] Specify embedding model during onboarding for the Knowledge Base (#218448)](#218448) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Søren Louv-Jansen","email":"soren.louv@elastic.co"},"sourceCommit":{"committedDate":"2025-03-20T09:59:59Z","message":"[Obs AI Assistant] Minor cleanup api tests (#215181)\n\n- Remove duplicated helpers\n- improve log messages\n- Clear intercepted requests after every test\n\n---------\n\nCo-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>","sha":"b9cd2c80404b0a0eb8ee51257ef01bdc1a71def6","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:skip","Team:Obs AI Assistant","ci:project-deploy-observability","v9.1.0"],"title":"[Obs AI Assistant] Minor cleanup api tests","number":215181,"url":"https://github.com/elastic/kibana/pull/215181","mergeCommit":{"message":"[Obs AI Assistant] Minor cleanup api tests (#215181)\n\n- Remove duplicated helpers\n- improve log messages\n- Clear intercepted requests after every test\n\n---------\n\nCo-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>","sha":"b9cd2c80404b0a0eb8ee51257ef01bdc1a71def6"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/215181","number":215181,"mergeCommit":{"message":"[Obs AI Assistant] Minor cleanup api tests (#215181)\n\n- Remove duplicated helpers\n- improve log messages\n- Clear intercepted requests after every test\n\n---------\n\nCo-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>","sha":"b9cd2c80404b0a0eb8ee51257ef01bdc1a71def6"}}]},{"author":{"name":"Søren Louv-Jansen","email":"soren.louv@elastic.co"},"sourceCommit":{"committedDate":"2025-04-08T08:13:00Z","message":"[Obs AI Assistant] Replace Task Manager usage with Lock Manager (#216916)\n\nThis PR replaces usage of the Kibana Task Manager with the [LockManager\nutility](#216397) that ensures\nstrict mutual exclusion when running operations such as re-indexing or\ndata migrations.\n\n---------\n\nCo-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"624410524f1a07dc72d92759464a3f4a85466c7a","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:skip","Team:Obs AI Assistant","v9.1.0","v8.19.0"],"title":"[Obs AI Assistant] Replace Task Manager usage with Lock Manager","number":216916,"url":"https://github.com/elastic/kibana/pull/216916","mergeCommit":{"message":"[Obs AI Assistant] Replace Task Manager usage with Lock Manager (#216916)\n\nThis PR replaces usage of the Kibana Task Manager with the [LockManager\nutility](#216397) that ensures\nstrict mutual exclusion when running operations such as re-indexing or\ndata migrations.\n\n---------\n\nCo-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"624410524f1a07dc72d92759464a3f4a85466c7a"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/216916","number":216916,"mergeCommit":{"message":"[Obs AI Assistant] Replace Task Manager usage with Lock Manager (#216916)\n\nThis PR replaces usage of the Kibana Task Manager with the [LockManager\nutility](#216397) that ensures\nstrict mutual exclusion when running operations such as re-indexing or\ndata migrations.\n\n---------\n\nCo-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"624410524f1a07dc72d92759464a3f4a85466c7a"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]},{"author":{"name":"Arturo Lidueña","email":"arturo.liduena@elastic.co"},"sourceCommit":{"committedDate":"2025-04-09T09:38:41Z","message":"update knowledge base installation flow (#214133)\n\n## Summary\nCloses [#209791](https://github.com/elastic/kibana/issues/209791)\nCloses [#204378](https://github.com/elastic/kibana/issues/204378)\n### Improve Knowledge Base Installation and Inference Endpoint\nManagement\n\n#### Enhancements \n- The **\"Install Knowledge Base\"** button is only shown when there is no\ninference endpoint. If the inference endpoint creation fails, an error\nnotification is displayed.\n- When an inference endpoint **exists and is ready**, no buttons or\nmessages are shown.\n- If the installation was in progress and **succeeded**, a success\nmessage is displayed.\n- If an inference endpoint is present, users can **inspect the state of\nthe model deployment** to diagnose issues.\n- When an inference endpoint exists but has an **error**, users can\nretry the installation.\n\n#### Inspection & Troubleshooting \nUsers can inspect the inference endpoint's deployment status, including:\n- **Model is deploying** \n- **No ML node available** \n- **Insufficient memory available** \n\n\nhttps://github.com/user-attachments/assets/03a91120-173c-4dc9-b6f2-59a8c8c8defb\n\n### Suggested Improvements \n- Display **autoscaling information** for better resource insights. \n- Show **ML node details** to help users understand the infrastructure.\n- Allow users to **install, deploy, and stop the model** directly from\nthe inspection popup.\n\nThese changes enhance visibility, troubleshooting, and control over\ninference endpoint deployment. 🚀\n\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [ ] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [ ]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials\n- [ ] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [ ] If a plugin configuration key changed, check if it needs to be\nallowlisted in the cloud and added to the [docker\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\n- [ ] This was checked for breaking HTTP API changes, and any breaking\nchanges have been approved by the breaking-change committee. The\n`release_note:breaking` label should be applied in these situations.\n- [ ] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [ ] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n### Identify risks\n\nDoes this PR introduce any risks? For example, consider risks like hard\nto test bugs, performance regression, potential of data loss.\n\nDescribe the risk, its severity, and mitigation for each identified\nrisk. Invite stakeholders and evaluate how to proceed before merging.\n\n- [ ] [See some risk\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n- [ ] ...","sha":"4109420186d586f91fd999cea0087bb8e06bfc59","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","backport missing","Team:Obs AI Assistant","backport:version","v9.1.0","v8.19.0"],"title":"update knowledge base installation flow","number":214133,"url":"https://github.com/elastic/kibana/pull/214133","mergeCommit":{"message":"update knowledge base installation flow (#214133)\n\n## Summary\nCloses [#209791](https://github.com/elastic/kibana/issues/209791)\nCloses [#204378](https://github.com/elastic/kibana/issues/204378)\n### Improve Knowledge Base Installation and Inference Endpoint\nManagement\n\n#### Enhancements \n- The **\"Install Knowledge Base\"** button is only shown when there is no\ninference endpoint. If the inference endpoint creation fails, an error\nnotification is displayed.\n- When an inference endpoint **exists and is ready**, no buttons or\nmessages are shown.\n- If the installation was in progress and **succeeded**, a success\nmessage is displayed.\n- If an inference endpoint is present, users can **inspect the state of\nthe model deployment** to diagnose issues.\n- When an inference endpoint exists but has an **error**, users can\nretry the installation.\n\n#### Inspection & Troubleshooting \nUsers can inspect the inference endpoint's deployment status, including:\n- **Model is deploying** \n- **No ML node available** \n- **Insufficient memory available** \n\n\nhttps://github.com/user-attachments/assets/03a91120-173c-4dc9-b6f2-59a8c8c8defb\n\n### Suggested Improvements \n- Display **autoscaling information** for better resource insights. \n- Show **ML node details** to help users understand the infrastructure.\n- Allow users to **install, deploy, and stop the model** directly from\nthe inspection popup.\n\nThese changes enhance visibility, troubleshooting, and control over\ninference endpoint deployment. 🚀\n\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [ ] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [ ]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials\n- [ ] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [ ] If a plugin configuration key changed, check if it needs to be\nallowlisted in the cloud and added to the [docker\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\n- [ ] This was checked for breaking HTTP API changes, and any breaking\nchanges have been approved by the breaking-change committee. The\n`release_note:breaking` label should be applied in these situations.\n- [ ] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [ ] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n### Identify risks\n\nDoes this PR introduce any risks? For example, consider risks like hard\nto test bugs, performance regression, potential of data loss.\n\nDescribe the risk, its severity, and mitigation for each identified\nrisk. Invite stakeholders and evaluate how to proceed before merging.\n\n- [ ] [See some risk\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n- [ ] ...","sha":"4109420186d586f91fd999cea0087bb8e06bfc59"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/214133","number":214133,"mergeCommit":{"message":"update knowledge base installation flow (#214133)\n\n## Summary\nCloses [#209791](https://github.com/elastic/kibana/issues/209791)\nCloses [#204378](https://github.com/elastic/kibana/issues/204378)\n### Improve Knowledge Base Installation and Inference Endpoint\nManagement\n\n#### Enhancements \n- The **\"Install Knowledge Base\"** button is only shown when there is no\ninference endpoint. If the inference endpoint creation fails, an error\nnotification is displayed.\n- When an inference endpoint **exists and is ready**, no buttons or\nmessages are shown.\n- If the installation was in progress and **succeeded**, a success\nmessage is displayed.\n- If an inference endpoint is present, users can **inspect the state of\nthe model deployment** to diagnose issues.\n- When an inference endpoint exists but has an **error**, users can\nretry the installation.\n\n#### Inspection & Troubleshooting \nUsers can inspect the inference endpoint's deployment status, including:\n- **Model is deploying** \n- **No ML node available** \n- **Insufficient memory available** \n\n\nhttps://github.com/user-attachments/assets/03a91120-173c-4dc9-b6f2-59a8c8c8defb\n\n### Suggested Improvements \n- Display **autoscaling information** for better resource insights. \n- Show **ML node details** to help users understand the infrastructure.\n- Allow users to **install, deploy, and stop the model** directly from\nthe inspection popup.\n\nThese changes enhance visibility, troubleshooting, and control over\ninference endpoint deployment. 🚀\n\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [ ] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [ ]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials\n- [ ] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [ ] If a plugin configuration key changed, check if it needs to be\nallowlisted in the cloud and added to the [docker\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\n- [ ] This was checked for breaking HTTP API changes, and any breaking\nchanges have been approved by the breaking-change committee. The\n`release_note:breaking` label should be applied in these situations.\n- [ ] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [ ] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n### Identify risks\n\nDoes this PR introduce any risks? For example, consider risks like hard\nto test bugs, performance regression, potential of data loss.\n\nDescribe the risk, its severity, and mitigation for each identified\nrisk. Invite stakeholders and evaluate how to proceed before merging.\n\n- [ ] [See some risk\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n- [ ] ...","sha":"4109420186d586f91fd999cea0087bb8e06bfc59"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]},{"author":{"name":"Viduni Wickramarachchi","email":"viduni.wickramarachchi@elastic.co"},"sourceCommit":{"committedDate":"2025-05-05T08:13:10Z","message":"[Obs AI Assistant] Specify embedding model during onboarding for the Knowledge Base (#218448)\n\nCloses https://github.com/elastic/obs-ai-assistant-team/issues/230\nCloses https://github.com/elastic/obs-ai-assistant-team/issues/232\n\nRelated to https://github.com/elastic/kibana/pull/215591\n\n## Summary\n\nThis PR implements the changes related to the first phase of supporing\nmultilingual Knowledge Base. The users have the ability to pick the\n`e5-small` model for the Knowledge Base, if they want support for\nlanguages other than English.\n\n<img width=\"610\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/4c815aa4-aa97-4845-98c5-e079dd92f23a\"\n/>\n\n<img width=\"1281\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/7c1bcd82-5464-497f-a053-7fe271da1cdd\"\n/>\n\n<img width=\"1280\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/bc084e90-c291-44ea-8560-e033729bfcca\"\n/>\n\nWhen the KB model is not allocated due to nodes scaling down:\n\n\n\n\n\n### Checklist\n\n- [x] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [ ] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n\n## Upgrade testing steps\n\n### 9.0 - 9.1 (main)\n\nCheckout `9.0` branch and start Kibana and ES. ES must be started with\n`path.data` to persist data:\n\n```\nyarn es snapshot --license trial --E path.data=/Users/sorenlouv/elastic/es_data/upgrade_test_9.0\n```\n\n---------\n\nCo-authored-by: Søren Louv-Jansen <soren.louv@elastic.co>\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by: Søren Louv-Jansen <sorenlouv@gmail.com>","sha":"dc019f85e19de28c1e2573108c47caa0080247c8","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:feature","Team:Obs AI Assistant","ci:project-deploy-observability","backport:version","v9.1.0","v8.19.0"],"title":"[Obs AI Assistant] Specify embedding model during onboarding for the Knowledge Base","number":218448,"url":"https://github.com/elastic/kibana/pull/218448","mergeCommit":{"message":"[Obs AI Assistant] Specify embedding model during onboarding for the Knowledge Base (#218448)\n\nCloses https://github.com/elastic/obs-ai-assistant-team/issues/230\nCloses https://github.com/elastic/obs-ai-assistant-team/issues/232\n\nRelated to https://github.com/elastic/kibana/pull/215591\n\n## Summary\n\nThis PR implements the changes related to the first phase of supporing\nmultilingual Knowledge Base. The users have the ability to pick the\n`e5-small` model for the Knowledge Base, if they want support for\nlanguages other than English.\n\n<img width=\"610\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/4c815aa4-aa97-4845-98c5-e079dd92f23a\"\n/>\n\n<img width=\"1281\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/7c1bcd82-5464-497f-a053-7fe271da1cdd\"\n/>\n\n<img width=\"1280\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/bc084e90-c291-44ea-8560-e033729bfcca\"\n/>\n\nWhen the KB model is not allocated due to nodes scaling down:\n\n\n\n\n\n### Checklist\n\n- [x] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [ ] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n\n## Upgrade testing steps\n\n### 9.0 - 9.1 (main)\n\nCheckout `9.0` branch and start Kibana and ES. ES must be started with\n`path.data` to persist data:\n\n```\nyarn es snapshot --license trial --E path.data=/Users/sorenlouv/elastic/es_data/upgrade_test_9.0\n```\n\n---------\n\nCo-authored-by: Søren Louv-Jansen <soren.louv@elastic.co>\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by: Søren Louv-Jansen <sorenlouv@gmail.com>","sha":"dc019f85e19de28c1e2573108c47caa0080247c8"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/218448","number":218448,"mergeCommit":{"message":"[Obs AI Assistant] Specify embedding model during onboarding for the Knowledge Base (#218448)\n\nCloses https://github.com/elastic/obs-ai-assistant-team/issues/230\nCloses https://github.com/elastic/obs-ai-assistant-team/issues/232\n\nRelated to https://github.com/elastic/kibana/pull/215591\n\n## Summary\n\nThis PR implements the changes related to the first phase of supporing\nmultilingual Knowledge Base. The users have the ability to pick the\n`e5-small` model for the Knowledge Base, if they want support for\nlanguages other than English.\n\n<img width=\"610\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/4c815aa4-aa97-4845-98c5-e079dd92f23a\"\n/>\n\n<img width=\"1281\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/7c1bcd82-5464-497f-a053-7fe271da1cdd\"\n/>\n\n<img width=\"1280\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/bc084e90-c291-44ea-8560-e033729bfcca\"\n/>\n\nWhen the KB model is not allocated due to nodes scaling down:\n\n\n\n\n\n### Checklist\n\n- [x] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [ ] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n\n## Upgrade testing steps\n\n### 9.0 - 9.1 (main)\n\nCheckout `9.0` branch and start Kibana and ES. ES must be started with\n`path.data` to persist data:\n\n```\nyarn es snapshot --license trial --E path.data=/Users/sorenlouv/elastic/es_data/upgrade_test_9.0\n```\n\n---------\n\nCo-authored-by: Søren Louv-Jansen <soren.louv@elastic.co>\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by: Søren Louv-Jansen <sorenlouv@gmail.com>","sha":"dc019f85e19de28c1e2573108c47caa0080247c8"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> --------- Co-authored-by: Arturo Lidueña <arturo.liduena@elastic.co> Co-authored-by: Viduni Wickramarachchi <viduni.wickramarachchi@elastic.co> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
akowalska622
pushed a commit
to akowalska622/kibana
that referenced
this pull request
May 29, 2025
Closes: elastic#218944 The index template for the Lock Manager was not created, causing index mappings and settings to be incorrect. Root cause: the function responsible for creating the index template (`ensureTemplatesAndIndexCreated`) was never invoked - only during automated testing 🤦 ## Manual testing steps The mappings for the lock manager index (`.kibana_locks-000001`) should match this: ```ts { mappings: { dynamic: false, properties: { token: { type: 'keyword' }, metadata: { enabled: false }, createdAt: { type: 'date' }, expiresAt: { type: 'date' }, }, }, } ``` ### Test 1 In this test we make sure that the LockManager library can recover and fix the mappings if the existing index has invalid mappings #### Delete existing assets if they already exist ``` DELETE .kibana_locks-000001 DELETE _index_template/.kibana_locks-index-template DELETE _component_template/.kibana_locks-component ``` #### Create lock index. This is invalid because it does not have the correct mappings ``` PUT .kibana_locks-000001 ``` (Restart Kibana) #### Check mappings are correct ``` GET .kibana_locks-000001/_mapping ``` ### Test 2 In this test we make sure that out of the box, the LockManager library creates an index with the correct mappings ``` DELETE .kibana_locks-000001 DELETE _index_template/.kibana_locks-index-template DELETE _component_template/.kibana_locks-component ``` (Restart Kibana) #### Check mappings are correct ``` GET .kibana_locks-000001/_mapping ``` Related: elastic#216916 elastic#216397 --------- Co-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR replaces usage of the Kibana Task Manager with the LockManager utility that ensures strict mutual exclusion when running operations such as re-indexing or data migrations.