Skip to content

[Security GenAI] Remove assistantNaturalLanguageESQLTool feature flag and enable by default#195480

Merged
stephmilovic merged 9 commits intoelastic:mainfrom
stephmilovic:esql_change
Oct 9, 2024
Merged

[Security GenAI] Remove assistantNaturalLanguageESQLTool feature flag and enable by default#195480
stephmilovic merged 9 commits intoelastic:mainfrom
stephmilovic:esql_change

Conversation

@stephmilovic
Copy link
Copy Markdown
Contributor

@stephmilovic stephmilovic commented Oct 8, 2024 •
edited by kibanamachine
Loading

Summary

After several weeks of evaluation testing, we feel confident to switch from ESQLKnowledgeBaseTool to NaturalLanguageESQLTool as our default ESQL tool. This PR takes the following steps to do so:

  • Removes the assistantNaturalLanguageESQLTool feature flag and enables the NaturalLanguageESQLTool by default
  • Removes the ESQLKnowledgeBaseTool
  • Removes the ESQL knowledge base doc loader loadESQL function as we no longer use them with the NaturalLanguageESQLTool
  • Removes esql_exists from isKnowledgeBaseSetup check
  • Removes knowledge base ascii docs from esql/documentation and esql/language_definition as NaturalLanguageESQLTool already has these references
  • Keeps knowledge base ascii docs from esql/example_queries as I am going to spike on including these in the NaturalLanguageESQLTool

To test

  1. Start with a fresh ES and Kibana
  2. Open assistant and click into knowledge base tab. Ensure that ESQL condition no longer appears (only ELSER Configured and Knowledge Base)
  3. Ask ESQL question. Check LangSmith trace to ensure NaturalLanguageESQLTool has been called.

Free feel to run the ESQL Generation Regression Suite but I've already done it for you!
Screenshot 2024-10-08 at 11 28 19 AM

@stephmilovic stephmilovic added release_note:enhancement v9.0.0 Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. backport:prev-minor Team:Security Generative AI Security Generative AI v8.16.0 labels Oct 8, 2024
@stephmilovic stephmilovic requested review from a team as code owners October 8, 2024 17:16
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Oct 8, 2024

Pinging @elastic/security-solution (Team: SecuritySolution)

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Oct 8, 2024

A documentation preview will be available soon.

Request a new doc build by commenting
  • Rebuild this PR: run docs-build
  • Rebuild this PR and all Elastic docs: run docs-build rebuild

run docs-build is much faster than run docs-build rebuild. A rebuild should only be needed in rare situations.

If your PR continues to fail for an unknown reason, the doc build pipeline may be broken. Elastic employees can check the pipeline status here.

@stephmilovic stephmilovic added the Feature:Security Assistant Security Assistant label Oct 8, 2024
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Oct 8, 2024

⏳ Build in-progress, with failures

Failed CI Steps

Test Failures

  • [job] [logs] Serverless Osquery Cypress Tests #8 / ALL - Live Query should validate the form should validate the form

History

@stephmilovic
Copy link
Copy Markdown
Contributor Author

stephmilovic commented Oct 8, 2024

@elasticmachine merge upstream

@stephmilovic
Copy link
Copy Markdown
Contributor Author

stephmilovic commented Oct 8, 2024

@elasticmachine merge upstream

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Oct 9, 2024

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
securitySolution 20.6MB 20.6MB -727.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
securitySolution 88.3KB 88.2KB -36.0B

History

@stephmilovic stephmilovic merged commit 3dd1ee8 into elastic:main Oct 9, 2024
@kibanamachine
Copy link
Copy Markdown
Contributor

kibanamachine commented Oct 9, 2024

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11256813292

@kibanamachine
Copy link
Copy Markdown
Contributor

kibanamachine commented Oct 9, 2024

💔 All backports failed

Status Branch Result
❌ 8.x Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 195480

Questions ?

Please refer to the Backport tool documentation

@kibanamachine kibanamachine mentioned this pull request Oct 9, 2024
Merged
@stephmilovic stephmilovic mentioned this pull request Oct 9, 2024
Merged
@stephmilovic
Copy link
Copy Markdown
Contributor Author

stephmilovic commented Oct 9, 2024

💚 All backports created successfully

Status Branch Result
✅ 8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

stephmilovic added a commit that referenced this pull request Oct 10, 2024
@stephmilovic
[8.x] [Security GenAI] Remove assistantNaturalLanguageESQLTool feat…
e92919e 
…ure flag and enable by default (#195480) (#195674)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Security GenAI] Remove `assistantNaturalLanguageESQLTool` feature
flag and enable by default
(#195480)](#195480)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Steph
Milovic","email":"stephanie.milovic@elastic.co"},"sourceCommit":{"committedDate":"2024-10-09T14:13:05Z","message":"[Security
GenAI] Remove `assistantNaturalLanguageESQLTool` feature flag and enable
by default
(#195480)","sha":"3dd1ee8ae00a40b582f2d1f784c96269a1d2e016","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement","v9.0.0","Team:
SecuritySolution","backport:prev-minor","Feature:Security
Assistant","Team:Security Generative
AI","v8.16.0"],"number":195480,"url":"https://github.com/elastic/kibana/pull/195480","mergeCommit":{"message":"[Security
GenAI] Remove `assistantNaturalLanguageESQLTool` feature flag and enable
by default
(#195480)","sha":"3dd1ee8ae00a40b582f2d1f784c96269a1d2e016"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/195480","number":195480,"mergeCommit":{"message":"[Security
GenAI] Remove `assistantNaturalLanguageESQLTool` feature flag and enable
by default
(#195480)","sha":"3dd1ee8ae00a40b582f2d1f784c96269a1d2e016"}},{"branch":"8.x","label":"v8.16.0","labelRegex":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@patrykkopycinski patrykkopycinski patrykkopycinski approved these changes

Assignees

No one assigned

Labels

Feature:Security Assistant Security Assistant release_note:enhancement Team:Security Generative AI Security Generative AI Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.16.0 v9.0.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@stephmilovic @elasticmachine @kibanamachine @patrykkopycinski