[Security Assistant] Add field type badge to Index Entry field suggestions #231904
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
spong
added
release_note:skip
Contributor
💛 Build succeeded, but was flaky
Failed CI Steps
Test Failures
Metrics [docs]Module Count
Async chunks
Unknown metric groupsESLint disabled in files
ESLint disabled line counts
Total ESLint disabled count
History
cc @spong |
KDKHD
reviewed
Aug 22, 2025
| "draw-graph": "node ./scripts/draw_graph", | ||
| "encode-security-labs-content": "node ./scripts/encode_security_labs_content" | ||
| "encode-security-labs-content": "node ./scripts/encode_security_labs_content", | ||
| "stress-test-mappings": "node ./scripts/stess_test_mappings" |
Member
There was a problem hiding this comment.
Thank you for adding this script!
Contributor
|
Starting backport for target branches: 9.1 https://github.com/elastic/kibana/actions/runs/17158313290 |
kibanamachine
added a commit
to kibanamachine/kibana
that referenced
this pull request
Aug 22, 2025
…tions (elastic#231904) ## Summary Small follow-up improvement to elastic#231376 which added support for `text` fields to Index Entries. This PR adds the field type as a badge in the suggestions so users will know if a semantic or lexical search will be performed (so they can adapt the query instructions accordingly). Note: Needed to update the field API request from `dataViews.getFieldsForWildcard` (which called `/internal/data_views/_fields_for_wildcard`) to use `/api/index_management/mapping/[indexName]` as the former did not have the option to include field type. I confirmed no new privileges were necessary for this API, and the user just needs the same index privileges as before. cc @jamesspi Field Options: <p align="center"> <img width="500" src="https://github.com/user-attachments/assets/f138c7f0-1d89-4946-8d27-fa6c9c49c60b" /> </p> Output Field Options: <p align="center"> <img width="500" src="https://github.com/user-attachments/assets/2b0395e5-d71d-43af-8a23-9bacc4b02b54" /> </p> --- As part of this PR I've also included the helper script from elastic#231376 for testing these large index/mapping scenarios. This script was almost entirely written in a collab session with `gemini-cli`, and is located in: > x-pack/solutions/security/plugins/elastic_assistant/scripts Options include: ``` bash Elasticsearch Index/Mapping Populator and Cleanup Script Usage: node stress_test_mappings.js [options] node stress_test_mappings.js --cleanup node stress_test_mappings.js --delete-by-count <number> Description: This script stress-tests an Elasticsearch instance by creating a large number of indices with many fields. It can also clean up the indices it creates. Creation Options: --host <url> Elasticsearch host URL (default: http://localhost:9200) --user <username> Username for basic auth (default: elastic) --pass <password> Password for basic auth (default: changeme) --apiKey <key> API key for authentication (overrides user/pass) --indices <number> Number of indices to create (default: 5000) --mappings <number> Number of mappings per index (default: 5000) --maxFields <number> The max number of fields per index (default: same as --mappings) --shards <number> Number of primary shards per index (default: 1) --replicas <number> Number of replicas per index (default: 0) Cleanup & Recovery Options: --cleanup Delete all indices created by this script. --delete-by-count <N> Delete the <N> newest stress-test indices. --yes Bypass confirmation prompt during cleanup. Other Options: -h, --help Show this help message ``` And some test executions are as follows. First CD into the assistant working directory: ``` cd x-pack/solutions/security/plugins/elastic_assistant/ ``` ##### Populate your local ES -- defaults to 5000 indices and 5000 mappings _per_ index. This _will cause_ a default local ES to crash, so stop early (~569), or change configuration :) ``` bash yarn stress-test-mappings ``` ##### If your ES is at its limits, you can slowly dial back the index count with the following: ``` bash yarn stress-test-mappings --delete-by-count 50 --yes ``` ##### Or cleanup all the indices you created entirely with: ``` bash yarn stress-test-mappings --cleanup --yes ``` ##### And for a cloud install, create an API key and populate with the following: ``` bash yarn stress-test-mappings -host https://stress-test.es.us-west2.gcp.elastic-cloud.com --apiKey APK_KEY_HERE ``` > [!IMPORTANT] > This is a quick utility script and may be buggy! Continue to vibe code it as you see fit, but it worked for my needs here for testing and validating this issue and fix 🙂 ### Checklist - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit 39a6983)
Contributor
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Aug 22, 2025
…suggestions (#231904) (#232674) # Backport This will backport the following commits from `main` to `9.1`: - [[Security Assistant] Add field type badge to Index Entry field suggestions (#231904)](#231904) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Garrett Spong","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-08-22T14:41:57Z","message":"[Security Assistant] Add field type badge to Index Entry field suggestions (#231904)\n\n## Summary\n\nSmall follow-up improvement to\nhttps://github.com//pull/231376 which added support for\n`text` fields to Index Entries. This PR adds the field type as a badge\nin the suggestions so users will know if a semantic or lexical search\nwill be performed (so they can adapt the query instructions\naccordingly).\n\n\nNote: Needed to update the field API request from\n`dataViews.getFieldsForWildcard` (which called\n`/internal/data_views/_fields_for_wildcard`) to use\n`/api/index_management/mapping/[indexName]` as the former did not have\nthe option to include field type. I confirmed no new privileges were\nnecessary for this API, and the user just needs the same index\nprivileges as before.\n\ncc @jamesspi \n\nField Options:\n<p align=\"center\">\n<img width=\"500\"\nsrc=\"https://github.com/user-attachments/assets/f138c7f0-1d89-4946-8d27-fa6c9c49c60b\"\n/>\n</p> \n\nOutput Field Options:\n<p align=\"center\">\n<img width=\"500\"\nsrc=\"https://github.com/user-attachments/assets/2b0395e5-d71d-43af-8a23-9bacc4b02b54\"\n/>\n</p> \n\n\n---\n\nAs part of this PR I've also included the helper script from\nhttps://github.com//pull/231376 for testing these large\nindex/mapping scenarios. This script was almost entirely written in a\ncollab session with `gemini-cli`, and is located in:\n\n> x-pack/solutions/security/plugins/elastic_assistant/scripts \n\nOptions include:\n\n``` bash\n Elasticsearch Index/Mapping Populator and Cleanup Script\n\n Usage:\n node stress_test_mappings.js [options]\n node stress_test_mappings.js --cleanup\n node stress_test_mappings.js --delete-by-count <number>\n\n Description:\n This script stress-tests an Elasticsearch instance by creating a large number\n of indices with many fields. It can also clean up the indices it creates.\n\n Creation Options:\n --host <url> Elasticsearch host URL (default: http://localhost:9200)\n --user <username> Username for basic auth (default: elastic)\n --pass <password> Password for basic auth (default: changeme)\n --apiKey <key> API key for authentication (overrides user/pass)\n --indices <number> Number of indices to create (default: 5000)\n --mappings <number> Number of mappings per index (default: 5000)\n --maxFields <number> The max number of fields per index (default: same as --mappings)\n --shards <number> Number of primary shards per index (default: 1)\n --replicas <number> Number of replicas per index (default: 0)\n\n Cleanup & Recovery Options:\n --cleanup Delete all indices created by this script.\n --delete-by-count <N> Delete the <N> newest stress-test indices.\n --yes Bypass confirmation prompt during cleanup.\n\n Other Options:\n -h, --help Show this help message\n```\n\n\nAnd some test executions are as follows. First CD into the assistant\nworking directory:\n\n```\ncd x-pack/solutions/security/plugins/elastic_assistant/\n```\n\n##### Populate your local ES -- defaults to 5000 indices and 5000\nmappings _per_ index. This _will cause_ a default local ES to crash, so\nstop early (~569), or change configuration :)\n``` bash\nyarn stress-test-mappings \n```\n\n##### If your ES is at its limits, you can slowly dial back the index\ncount with the following:\n``` bash\nyarn stress-test-mappings --delete-by-count 50 --yes\n```\n\n##### Or cleanup all the indices you created entirely with:\n``` bash\nyarn stress-test-mappings --cleanup --yes\n```\n\n##### And for a cloud install, create an API key and populate with the\nfollowing:\n``` bash\nyarn stress-test-mappings -host https://stress-test.es.us-west2.gcp.elastic-cloud.com --apiKey APK_KEY_HERE\n```\n\n> [!IMPORTANT]\n> This is a quick utility script and may be buggy! Continue to vibe code\nit as you see fit, but it worked for my needs here for testing and\nvalidating this issue and fix 🙂\n\n\n\n\n### Checklist\n\n- [X] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n\n---------\n\nCo-authored-by: kibanamachine <[email protected]>","sha":"39a6983ded36b572879346bbcfada819156f3e11","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Security Generative AI","backport:version","v9.2.0","v9.1.3"],"title":"[Security Assistant] Add field type badge to Index Entry field suggestions","number":231904,"url":"https://github.com/elastic/kibana/pull/231904","mergeCommit":{"message":"[Security Assistant] Add field type badge to Index Entry field suggestions (#231904)\n\n## Summary\n\nSmall follow-up improvement to\nhttps://github.com//pull/231376 which added support for\n`text` fields to Index Entries. This PR adds the field type as a badge\nin the suggestions so users will know if a semantic or lexical search\nwill be performed (so they can adapt the query instructions\naccordingly).\n\n\nNote: Needed to update the field API request from\n`dataViews.getFieldsForWildcard` (which called\n`/internal/data_views/_fields_for_wildcard`) to use\n`/api/index_management/mapping/[indexName]` as the former did not have\nthe option to include field type. I confirmed no new privileges were\nnecessary for this API, and the user just needs the same index\nprivileges as before.\n\ncc @jamesspi \n\nField Options:\n<p align=\"center\">\n<img width=\"500\"\nsrc=\"https://github.com/user-attachments/assets/f138c7f0-1d89-4946-8d27-fa6c9c49c60b\"\n/>\n</p> \n\nOutput Field Options:\n<p align=\"center\">\n<img width=\"500\"\nsrc=\"https://github.com/user-attachments/assets/2b0395e5-d71d-43af-8a23-9bacc4b02b54\"\n/>\n</p> \n\n\n---\n\nAs part of this PR I've also included the helper script from\nhttps://github.com//pull/231376 for testing these large\nindex/mapping scenarios. This script was almost entirely written in a\ncollab session with `gemini-cli`, and is located in:\n\n> x-pack/solutions/security/plugins/elastic_assistant/scripts \n\nOptions include:\n\n``` bash\n Elasticsearch Index/Mapping Populator and Cleanup Script\n\n Usage:\n node stress_test_mappings.js [options]\n node stress_test_mappings.js --cleanup\n node stress_test_mappings.js --delete-by-count <number>\n\n Description:\n This script stress-tests an Elasticsearch instance by creating a large number\n of indices with many fields. It can also clean up the indices it creates.\n\n Creation Options:\n --host <url> Elasticsearch host URL (default: http://localhost:9200)\n --user <username> Username for basic auth (default: elastic)\n --pass <password> Password for basic auth (default: changeme)\n --apiKey <key> API key for authentication (overrides user/pass)\n --indices <number> Number of indices to create (default: 5000)\n --mappings <number> Number of mappings per index (default: 5000)\n --maxFields <number> The max number of fields per index (default: same as --mappings)\n --shards <number> Number of primary shards per index (default: 1)\n --replicas <number> Number of replicas per index (default: 0)\n\n Cleanup & Recovery Options:\n --cleanup Delete all indices created by this script.\n --delete-by-count <N> Delete the <N> newest stress-test indices.\n --yes Bypass confirmation prompt during cleanup.\n\n Other Options:\n -h, --help Show this help message\n```\n\n\nAnd some test executions are as follows. First CD into the assistant\nworking directory:\n\n```\ncd x-pack/solutions/security/plugins/elastic_assistant/\n```\n\n##### Populate your local ES -- defaults to 5000 indices and 5000\nmappings _per_ index. This _will cause_ a default local ES to crash, so\nstop early (~569), or change configuration :)\n``` bash\nyarn stress-test-mappings \n```\n\n##### If your ES is at its limits, you can slowly dial back the index\ncount with the following:\n``` bash\nyarn stress-test-mappings --delete-by-count 50 --yes\n```\n\n##### Or cleanup all the indices you created entirely with:\n``` bash\nyarn stress-test-mappings --cleanup --yes\n```\n\n##### And for a cloud install, create an API key and populate with the\nfollowing:\n``` bash\nyarn stress-test-mappings -host https://stress-test.es.us-west2.gcp.elastic-cloud.com --apiKey APK_KEY_HERE\n```\n\n> [!IMPORTANT]\n> This is a quick utility script and may be buggy! Continue to vibe code\nit as you see fit, but it worked for my needs here for testing and\nvalidating this issue and fix 🙂\n\n\n\n\n### Checklist\n\n- [X] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n\n---------\n\nCo-authored-by: kibanamachine <[email protected]>","sha":"39a6983ded36b572879346bbcfada819156f3e11"}},"sourceBranch":"main","suggestedTargetBranches":["9.1"],"targetPullRequestStates":[{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/231904","number":231904,"mergeCommit":{"message":"[Security Assistant] Add field type badge to Index Entry field suggestions (#231904)\n\n## Summary\n\nSmall follow-up improvement to\nhttps://github.com//pull/231376 which added support for\n`text` fields to Index Entries. This PR adds the field type as a badge\nin the suggestions so users will know if a semantic or lexical search\nwill be performed (so they can adapt the query instructions\naccordingly).\n\n\nNote: Needed to update the field API request from\n`dataViews.getFieldsForWildcard` (which called\n`/internal/data_views/_fields_for_wildcard`) to use\n`/api/index_management/mapping/[indexName]` as the former did not have\nthe option to include field type. I confirmed no new privileges were\nnecessary for this API, and the user just needs the same index\nprivileges as before.\n\ncc @jamesspi \n\nField Options:\n<p align=\"center\">\n<img width=\"500\"\nsrc=\"https://github.com/user-attachments/assets/f138c7f0-1d89-4946-8d27-fa6c9c49c60b\"\n/>\n</p> \n\nOutput Field Options:\n<p align=\"center\">\n<img width=\"500\"\nsrc=\"https://github.com/user-attachments/assets/2b0395e5-d71d-43af-8a23-9bacc4b02b54\"\n/>\n</p> \n\n\n---\n\nAs part of this PR I've also included the helper script from\nhttps://github.com//pull/231376 for testing these large\nindex/mapping scenarios. This script was almost entirely written in a\ncollab session with `gemini-cli`, and is located in:\n\n> x-pack/solutions/security/plugins/elastic_assistant/scripts \n\nOptions include:\n\n``` bash\n Elasticsearch Index/Mapping Populator and Cleanup Script\n\n Usage:\n node stress_test_mappings.js [options]\n node stress_test_mappings.js --cleanup\n node stress_test_mappings.js --delete-by-count <number>\n\n Description:\n This script stress-tests an Elasticsearch instance by creating a large number\n of indices with many fields. It can also clean up the indices it creates.\n\n Creation Options:\n --host <url> Elasticsearch host URL (default: http://localhost:9200)\n --user <username> Username for basic auth (default: elastic)\n --pass <password> Password for basic auth (default: changeme)\n --apiKey <key> API key for authentication (overrides user/pass)\n --indices <number> Number of indices to create (default: 5000)\n --mappings <number> Number of mappings per index (default: 5000)\n --maxFields <number> The max number of fields per index (default: same as --mappings)\n --shards <number> Number of primary shards per index (default: 1)\n --replicas <number> Number of replicas per index (default: 0)\n\n Cleanup & Recovery Options:\n --cleanup Delete all indices created by this script.\n --delete-by-count <N> Delete the <N> newest stress-test indices.\n --yes Bypass confirmation prompt during cleanup.\n\n Other Options:\n -h, --help Show this help message\n```\n\n\nAnd some test executions are as follows. First CD into the assistant\nworking directory:\n\n```\ncd x-pack/solutions/security/plugins/elastic_assistant/\n```\n\n##### Populate your local ES -- defaults to 5000 indices and 5000\nmappings _per_ index. This _will cause_ a default local ES to crash, so\nstop early (~569), or change configuration :)\n``` bash\nyarn stress-test-mappings \n```\n\n##### If your ES is at its limits, you can slowly dial back the index\ncount with the following:\n``` bash\nyarn stress-test-mappings --delete-by-count 50 --yes\n```\n\n##### Or cleanup all the indices you created entirely with:\n``` bash\nyarn stress-test-mappings --cleanup --yes\n```\n\n##### And for a cloud install, create an API key and populate with the\nfollowing:\n``` bash\nyarn stress-test-mappings -host https://stress-test.es.us-west2.gcp.elastic-cloud.com --apiKey APK_KEY_HERE\n```\n\n> [!IMPORTANT]\n> This is a quick utility script and may be buggy! Continue to vibe code\nit as you see fit, but it worked for my needs here for testing and\nvalidating this issue and fix 🙂\n\n\n\n\n### Checklist\n\n- [X] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n\n---------\n\nCo-authored-by: kibanamachine <[email protected]>","sha":"39a6983ded36b572879346bbcfada819156f3e11"}},{"branch":"9.1","label":"v9.1.3","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Garrett Spong <[email protected]>
qn895
pushed a commit
to qn895/kibana
that referenced
this pull request
Aug 26, 2025
…tions (elastic#231904) ## Summary Small follow-up improvement to elastic#231376 which added support for `text` fields to Index Entries. This PR adds the field type as a badge in the suggestions so users will know if a semantic or lexical search will be performed (so they can adapt the query instructions accordingly). Note: Needed to update the field API request from `dataViews.getFieldsForWildcard` (which called `/internal/data_views/_fields_for_wildcard`) to use `/api/index_management/mapping/[indexName]` as the former did not have the option to include field type. I confirmed no new privileges were necessary for this API, and the user just needs the same index privileges as before. cc @jamesspi Field Options: <p align="center"> <img width="500" src="https://github.com/user-attachments/assets/f138c7f0-1d89-4946-8d27-fa6c9c49c60b" /> </p> Output Field Options: <p align="center"> <img width="500" src="https://github.com/user-attachments/assets/2b0395e5-d71d-43af-8a23-9bacc4b02b54" /> </p> --- As part of this PR I've also included the helper script from elastic#231376 for testing these large index/mapping scenarios. This script was almost entirely written in a collab session with `gemini-cli`, and is located in: > x-pack/solutions/security/plugins/elastic_assistant/scripts Options include: ``` bash Elasticsearch Index/Mapping Populator and Cleanup Script Usage: node stress_test_mappings.js [options] node stress_test_mappings.js --cleanup node stress_test_mappings.js --delete-by-count <number> Description: This script stress-tests an Elasticsearch instance by creating a large number of indices with many fields. It can also clean up the indices it creates. Creation Options: --host <url> Elasticsearch host URL (default: http://localhost:9200) --user <username> Username for basic auth (default: elastic) --pass <password> Password for basic auth (default: changeme) --apiKey <key> API key for authentication (overrides user/pass) --indices <number> Number of indices to create (default: 5000) --mappings <number> Number of mappings per index (default: 5000) --maxFields <number> The max number of fields per index (default: same as --mappings) --shards <number> Number of primary shards per index (default: 1) --replicas <number> Number of replicas per index (default: 0) Cleanup & Recovery Options: --cleanup Delete all indices created by this script. --delete-by-count <N> Delete the <N> newest stress-test indices. --yes Bypass confirmation prompt during cleanup. Other Options: -h, --help Show this help message ``` And some test executions are as follows. First CD into the assistant working directory: ``` cd x-pack/solutions/security/plugins/elastic_assistant/ ``` ##### Populate your local ES -- defaults to 5000 indices and 5000 mappings _per_ index. This _will cause_ a default local ES to crash, so stop early (~569), or change configuration :) ``` bash yarn stress-test-mappings ``` ##### If your ES is at its limits, you can slowly dial back the index count with the following: ``` bash yarn stress-test-mappings --delete-by-count 50 --yes ``` ##### Or cleanup all the indices you created entirely with: ``` bash yarn stress-test-mappings --cleanup --yes ``` ##### And for a cloud install, create an API key and populate with the following: ``` bash yarn stress-test-mappings -host https://stress-test.es.us-west2.gcp.elastic-cloud.com --apiKey APK_KEY_HERE ``` > [!IMPORTANT] > This is a quick utility script and may be buggy! Continue to vibe code it as you see fit, but it worked for my needs here for testing and validating this issue and fix 🙂 ### Checklist - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <[email protected]>
KodeRad
pushed a commit
to KodeRad/kibana
that referenced
this pull request
Aug 28, 2025
…tions (elastic#231904) ## Summary Small follow-up improvement to elastic#231376 which added support for `text` fields to Index Entries. This PR adds the field type as a badge in the suggestions so users will know if a semantic or lexical search will be performed (so they can adapt the query instructions accordingly). Note: Needed to update the field API request from `dataViews.getFieldsForWildcard` (which called `/internal/data_views/_fields_for_wildcard`) to use `/api/index_management/mapping/[indexName]` as the former did not have the option to include field type. I confirmed no new privileges were necessary for this API, and the user just needs the same index privileges as before. cc @jamesspi Field Options: <p align="center"> <img width="500" src="https://github.com/user-attachments/assets/f138c7f0-1d89-4946-8d27-fa6c9c49c60b" /> </p> Output Field Options: <p align="center"> <img width="500" src="https://github.com/user-attachments/assets/2b0395e5-d71d-43af-8a23-9bacc4b02b54" /> </p> --- As part of this PR I've also included the helper script from elastic#231376 for testing these large index/mapping scenarios. This script was almost entirely written in a collab session with `gemini-cli`, and is located in: > x-pack/solutions/security/plugins/elastic_assistant/scripts Options include: ``` bash Elasticsearch Index/Mapping Populator and Cleanup Script Usage: node stress_test_mappings.js [options] node stress_test_mappings.js --cleanup node stress_test_mappings.js --delete-by-count <number> Description: This script stress-tests an Elasticsearch instance by creating a large number of indices with many fields. It can also clean up the indices it creates. Creation Options: --host <url> Elasticsearch host URL (default: http://localhost:9200) --user <username> Username for basic auth (default: elastic) --pass <password> Password for basic auth (default: changeme) --apiKey <key> API key for authentication (overrides user/pass) --indices <number> Number of indices to create (default: 5000) --mappings <number> Number of mappings per index (default: 5000) --maxFields <number> The max number of fields per index (default: same as --mappings) --shards <number> Number of primary shards per index (default: 1) --replicas <number> Number of replicas per index (default: 0) Cleanup & Recovery Options: --cleanup Delete all indices created by this script. --delete-by-count <N> Delete the <N> newest stress-test indices. --yes Bypass confirmation prompt during cleanup. Other Options: -h, --help Show this help message ``` And some test executions are as follows. First CD into the assistant working directory: ``` cd x-pack/solutions/security/plugins/elastic_assistant/ ``` ##### Populate your local ES -- defaults to 5000 indices and 5000 mappings _per_ index. This _will cause_ a default local ES to crash, so stop early (~569), or change configuration :) ``` bash yarn stress-test-mappings ``` ##### If your ES is at its limits, you can slowly dial back the index count with the following: ``` bash yarn stress-test-mappings --delete-by-count 50 --yes ``` ##### Or cleanup all the indices you created entirely with: ``` bash yarn stress-test-mappings --cleanup --yes ``` ##### And for a cloud install, create an API key and populate with the following: ``` bash yarn stress-test-mappings -host https://stress-test.es.us-west2.gcp.elastic-cloud.com --apiKey APK_KEY_HERE ``` > [!IMPORTANT] > This is a quick utility script and may be buggy! Continue to vibe code it as you see fit, but it worked for my needs here for testing and validating this issue and fix 🙂 ### Checklist - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Small follow-up improvement to #231376 which added support for
textfields to Index Entries. This PR adds the field type as a badge in the suggestions so users will know if a semantic or lexical search will be performed (so they can adapt the query instructions accordingly).Note: Needed to update the field API request from
dataViews.getFieldsForWildcard(which called/internal/data_views/_fields_for_wildcard) to use/api/index_management/mapping/[indexName]as the former did not have the option to include field type. I confirmed no new privileges were necessary for this API, and the user just needs the same index privileges as before.cc @jamesspi
Field Options:
Output Field Options:
As part of this PR I've also included the helper script from #231376 for testing these large index/mapping scenarios. This script was almost entirely written in a collab session with
gemini-cli, and is located in:Options include:
Elasticsearch Index/Mapping Populator and Cleanup Script Usage: node stress_test_mappings.js [options] node stress_test_mappings.js --cleanup node stress_test_mappings.js --delete-by-count <number> Description: This script stress-tests an Elasticsearch instance by creating a large number of indices with many fields. It can also clean up the indices it creates. Creation Options: --host <url> Elasticsearch host URL (default: http://localhost:9200) --user <username> Username for basic auth (default: elastic) --pass <password> Password for basic auth (default: changeme) --apiKey <key> API key for authentication (overrides user/pass) --indices <number> Number of indices to create (default: 5000) --mappings <number> Number of mappings per index (default: 5000) --maxFields <number> The max number of fields per index (default: same as --mappings) --shards <number> Number of primary shards per index (default: 1) --replicas <number> Number of replicas per index (default: 0) Cleanup & Recovery Options: --cleanup Delete all indices created by this script. --delete-by-count <N> Delete the <N> newest stress-test indices. --yes Bypass confirmation prompt during cleanup. Other Options: -h, --help Show this help messageAnd some test executions are as follows. First CD into the assistant working directory:
Populate your local ES -- defaults to 5000 indices and 5000 mappings per index. This will cause a default local ES to crash, so stop early (~569), or change configuration :)
If your ES is at its limits, you can slowly dial back the index count with the following:
Or cleanup all the indices you created entirely with:
And for a cloud install, create an API key and populate with the following:
Important
This is a quick utility script and may be buggy! Continue to vibe code it as you see fit, but it worked for my needs here for testing and validating this issue and fix 🙂
Checklist