[Security Solution] [Elastic AI Assistant] Adds internal `Get Evaluate` API and migrates `Post Evaluate` API to OAS by spong · Pull Request #175338 · elastic/kibana

spong · 2024-01-23T18:00:04Z

Summary

In #174317 we added support for OpenAPI codegen, this PR builds on that functionality by migrating the Post Evaluate route /internal/elastic_assistant/evaluate to be backed by an OAS, and adds a basic Get Evaluate route for rounding out the enhancements outlined in https://github.com/elastic/security-team/issues/8167 (to be in a subsequent PR).

Changes include:

Migration of Post Evaluate route to OAS
Migration of Post Evaluate route to use versioned router
Extracted evaluate API calls from x-pack/packages/kbn-elastic-assistant/impl/assistant/api/api.tsx to x-pack/packages/kbn-elastic-assistant/impl/assistant/api/evaluate/evaluate.tsx
- Co-located relevant use_perform_evaluation hook
Adds Get Evaluate route, and corresponding use_evaluation_data hook. Currently only returns agentExecutors to be selected for evaluation.
API versioning constants added to x-pack/packages/kbn-elastic-assistant-common/impl/schemas/index.ts
Adds new buildRouteValidationWithZod function to x-pack/plugins/elastic_assistant/server/schemas/common.ts for validating routes against OAS generated zod schemas.

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Unit or functional tests were updated or added to match the most common scenarios

elasticmachine · 2024-01-23T18:00:07Z

Pinging @elastic/security-solution (Team: SecuritySolution)

…fix'

spong · 2024-01-29T19:05:19Z

...packages/kbn-elastic-assistant-common/impl/schemas/evaluation/get_evaluate_route.schema.yaml

+  title: Get Evaluate API endpoint
+  version: '1'
+paths:
+  /internal/elastic_assistant/evaluate:


Note: I re-used the existing evaluate route as that seemed the most ergonomic for fetching metadata that would be useful for executing an eval cycle via POST /evaluate.

I have no plan of exposing a 'get evaluation results' endpoint at the moment (since we've been treating that data as ephemeral), so wasn't worried about any overlap here. This is an internal API, so there is flexibility and it can change as needed.

kibana-ci · 2024-01-29T19:44:26Z

💚 Build Succeeded

Buildkite Build
Commit: c4b569f

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`securitySolution`	4954	4959	+5

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/elastic-assistant-common`	32	57	+25

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`securitySolution`	11.2MB	11.2MB	+3.0KB

Unknown metric groups

API count

id	before	after	diff
`@kbn/elastic-assistant-common`	34	59	+25

History

💚 Build #189485 succeeded df1a836
💔 Build #189466 failed 6bd9426
💛 Build #188823 was flaky bcce197
💔 Build #188806 failed bb6a8f8

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @spong

stephmilovic

Thanks for the updates. Reviewed, pulled, and tested, LGTM!

… Evaluate` API and migrates `Post Evaluate` API to OAS (elastic#175338)" This reverts commit 38f0a7a.

…t Evaluate` API and migrates `Post Evaluate` API to OAS (elastic#175338)" This reverts commit b201bf6.

… Evaluate` API and migrates `Post Evaluate` API to OAS (#175338)" This reverts commit 38f0a7a.

jbudz · 2024-01-31T19:37:33Z

We reverted this due to an increase in jest errors in x-pack/plugins/security_solution/public/common/jest.config.js. Builds available at #176005

… API and migrates Post Evaluate API to OAS (#176025) > [!IMPORTANT] > This PR is a reintroduction of #175338, which was [reverted](#175338 (comment)) due to sporadic jest failures within the `security_solution` plugin. Root cause was identified and detailed in #176005 (comment). ## Summary In #174317 we added support for OpenAPI codegen, this PR builds on that functionality by migrating the `Post Evaluate` route `/internal/elastic_assistant/evaluate` to be backed by an OAS, and adds a basic `Get Evaluate` route for rounding out the enhancements outlined in elastic/security-team#8167 (to be in a subsequent PR). Changes include: * Migration of `Post Evaluate` route to OAS * Migration of `Post Evaluate` route to use versioned router * Extracted `evaluate` API calls from `x-pack/packages/kbn-elastic-assistant/impl/assistant/api/api.tsx` to `x-pack/packages/kbn-elastic-assistant/impl/assistant/api/evaluate/evaluate.tsx` * Co-located relevant `use_perform_evaluation` hook * Adds `Get Evaluate` route, and corresponding `use_evaluation_data` hook. Currently only returns `agentExecutors` to be selected for evaluation. * API versioning constants added to `x-pack/packages/kbn-elastic-assistant-common/impl/schemas/index.ts` * Adds new `buildRouteValidationWithZod` function to `x-pack/plugins/elastic_assistant/server/schemas/common.ts` for validating routes against OAS generated zod schemas. ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

… API and migrates Post Evaluate API to OAS (elastic#176025) > [!IMPORTANT] > This PR is a reintroduction of elastic#175338, which was [reverted](elastic#175338 (comment)) due to sporadic jest failures within the `security_solution` plugin. Root cause was identified and detailed in elastic#176005 (comment). ## Summary In elastic#174317 we added support for OpenAPI codegen, this PR builds on that functionality by migrating the `Post Evaluate` route `/internal/elastic_assistant/evaluate` to be backed by an OAS, and adds a basic `Get Evaluate` route for rounding out the enhancements outlined in elastic/security-team#8167 (to be in a subsequent PR). Changes include: * Migration of `Post Evaluate` route to OAS * Migration of `Post Evaluate` route to use versioned router * Extracted `evaluate` API calls from `x-pack/packages/kbn-elastic-assistant/impl/assistant/api/api.tsx` to `x-pack/packages/kbn-elastic-assistant/impl/assistant/api/evaluate/evaluate.tsx` * Co-located relevant `use_perform_evaluation` hook * Adds `Get Evaluate` route, and corresponding `use_evaluation_data` hook. Currently only returns `agentExecutors` to be selected for evaluation. * API versioning constants added to `x-pack/packages/kbn-elastic-assistant-common/impl/schemas/index.ts` * Adds new `buildRouteValidationWithZod` function to `x-pack/plugins/elastic_assistant/server/schemas/common.ts` for validating routes against OAS generated zod schemas. ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

…e` API and migrates `Post Evaluate` API to OAS (elastic#175338) ## Summary In elastic#174317 we added support for OpenAPI codegen, this PR builds on that functionality by migrating the `Post Evaluate` route `/internal/elastic_assistant/evaluate` to be backed by an OAS, and adds a basic `Get Evaluate` route for rounding out the enhancements outlined in elastic/security-team#8167 (to be in a subsequent PR). Changes include: * Migration of `Post Evaluate` route to OAS * Migration of `Post Evaluate` route to use versioned router * Extracted `evaluate` API calls from `x-pack/packages/kbn-elastic-assistant/impl/assistant/api/api.tsx` to `x-pack/packages/kbn-elastic-assistant/impl/assistant/api/evaluate/evaluate.tsx` * Co-located relevant `use_perform_evaluation` hook * Adds `Get Evaluate` route, and corresponding `use_evaluation_data` hook. Currently only returns `agentExecutors` to be selected for evaluation. * API versioning constants added to `x-pack/packages/kbn-elastic-assistant-common/impl/schemas/index.ts` * Adds new `buildRouteValidationWithZod` function to `x-pack/plugins/elastic_assistant/server/schemas/common.ts` for validating routes against OAS generated zod schemas. ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

… Evaluate` API and migrates `Post Evaluate` API to OAS (elastic#175338)" This reverts commit 38f0a7a.

… API and migrates Post Evaluate API to OAS (elastic#176025) > [!IMPORTANT] > This PR is a reintroduction of elastic#175338, which was [reverted](elastic#175338 (comment)) due to sporadic jest failures within the `security_solution` plugin. Root cause was identified and detailed in elastic#176005 (comment). ## Summary In elastic#174317 we added support for OpenAPI codegen, this PR builds on that functionality by migrating the `Post Evaluate` route `/internal/elastic_assistant/evaluate` to be backed by an OAS, and adds a basic `Get Evaluate` route for rounding out the enhancements outlined in elastic/security-team#8167 (to be in a subsequent PR). Changes include: * Migration of `Post Evaluate` route to OAS * Migration of `Post Evaluate` route to use versioned router * Extracted `evaluate` API calls from `x-pack/packages/kbn-elastic-assistant/impl/assistant/api/api.tsx` to `x-pack/packages/kbn-elastic-assistant/impl/assistant/api/evaluate/evaluate.tsx` * Co-located relevant `use_perform_evaluation` hook * Adds `Get Evaluate` route, and corresponding `use_evaluation_data` hook. Currently only returns `agentExecutors` to be selected for evaluation. * API versioning constants added to `x-pack/packages/kbn-elastic-assistant-common/impl/schemas/index.ts` * Adds new `buildRouteValidationWithZod` function to `x-pack/plugins/elastic_assistant/server/schemas/common.ts` for validating routes against OAS generated zod schemas. ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

…e` API and migrates `Post Evaluate` API to OAS (elastic#175338) ## Summary In elastic#174317 we added support for OpenAPI codegen, this PR builds on that functionality by migrating the `Post Evaluate` route `/internal/elastic_assistant/evaluate` to be backed by an OAS, and adds a basic `Get Evaluate` route for rounding out the enhancements outlined in elastic/security-team#8167 (to be in a subsequent PR). Changes include: * Migration of `Post Evaluate` route to OAS * Migration of `Post Evaluate` route to use versioned router * Extracted `evaluate` API calls from `x-pack/packages/kbn-elastic-assistant/impl/assistant/api/api.tsx` to `x-pack/packages/kbn-elastic-assistant/impl/assistant/api/evaluate/evaluate.tsx` * Co-located relevant `use_perform_evaluation` hook * Adds `Get Evaluate` route, and corresponding `use_evaluation_data` hook. Currently only returns `agentExecutors` to be selected for evaluation. * API versioning constants added to `x-pack/packages/kbn-elastic-assistant-common/impl/schemas/index.ts` * Adds new `buildRouteValidationWithZod` function to `x-pack/plugins/elastic_assistant/server/schemas/common.ts` for validating routes against OAS generated zod schemas. ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

… Evaluate` API and migrates `Post Evaluate` API to OAS (elastic#175338)" This reverts commit 38f0a7a.

… API and migrates Post Evaluate API to OAS (elastic#176025) > [!IMPORTANT] > This PR is a reintroduction of elastic#175338, which was [reverted](elastic#175338 (comment)) due to sporadic jest failures within the `security_solution` plugin. Root cause was identified and detailed in elastic#176005 (comment). ## Summary In elastic#174317 we added support for OpenAPI codegen, this PR builds on that functionality by migrating the `Post Evaluate` route `/internal/elastic_assistant/evaluate` to be backed by an OAS, and adds a basic `Get Evaluate` route for rounding out the enhancements outlined in elastic/security-team#8167 (to be in a subsequent PR). Changes include: * Migration of `Post Evaluate` route to OAS * Migration of `Post Evaluate` route to use versioned router * Extracted `evaluate` API calls from `x-pack/packages/kbn-elastic-assistant/impl/assistant/api/api.tsx` to `x-pack/packages/kbn-elastic-assistant/impl/assistant/api/evaluate/evaluate.tsx` * Co-located relevant `use_perform_evaluation` hook * Adds `Get Evaluate` route, and corresponding `use_evaluation_data` hook. Currently only returns `agentExecutors` to be selected for evaluation. * API versioning constants added to `x-pack/packages/kbn-elastic-assistant-common/impl/schemas/index.ts` * Adds new `buildRouteValidationWithZod` function to `x-pack/plugins/elastic_assistant/server/schemas/common.ts` for validating routes against OAS generated zod schemas. ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

spong added 2 commits January 23, 2024 10:45

Migrates post evaluate API to OAS and adds new get evaluate API

f12be81

Merge branch 'main' of github.com:elastic/kibana into eval+++

b686fd5

spong added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Feature:Security Assistant Security Assistant v8.13.0 labels Jan 23, 2024

spong self-assigned this Jan 23, 2024

spong requested a review from a team as a code owner January 23, 2024 18:00

kibanamachine and others added 4 commits January 23, 2024 18:06

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

bb6a8f8

…fix'

Linter fix

bcce197

Merge branch 'main' into eval+++

6bd9426

Merge branch 'main' into eval+++

df1a836

spong mentioned this pull request Jan 29, 2024

[Security Solution] Security AI Assistant persistent storage. #173487

Merged

Merge branch 'main' into eval+++

c4b569f

spong requested a review from a team as a code owner January 29, 2024 18:27

spong commented Jan 29, 2024

View reviewed changes

stephmilovic approved these changes Jan 30, 2024

View reviewed changes

spong merged commit 38f0a7a into elastic:main Jan 30, 2024

spong deleted the eval+++ branch January 30, 2024 23:03

jbudz added a commit to jbudz/kibana that referenced this pull request Jan 31, 2024

Revert "[Security Solution] [Elastic AI Assistant] Adds internal `Get…

b201bf6

… Evaluate` API and migrates `Post Evaluate` API to OAS (elastic#175338)" This reverts commit 38f0a7a.

jbudz added a commit to jbudz/kibana that referenced this pull request Jan 31, 2024

Reapply "[Security Solution] [Elastic AI Assistant] Adds internal `Ge…

b39cc87

…t Evaluate` API and migrates `Post Evaluate` API to OAS (elastic#175338)" This reverts commit b201bf6.

jbudz added a commit that referenced this pull request Jan 31, 2024

Revert "[Security Solution] [Elastic AI Assistant] Adds internal `Get…

f8f6913

… Evaluate` API and migrates `Post Evaluate` API to OAS (#175338)" This reverts commit 38f0a7a.

jbudz added the reverted label Jan 31, 2024

jbudz mentioned this pull request Jan 31, 2024

Failing test: x-pack/plugins/security_solution/public/common/components/charts/barchart.test.tsx #175984

Closed

spong mentioned this pull request Jan 31, 2024

Jest flakiness testing #176005

Closed

spong restored the eval+++ branch January 31, 2024 22:13

spong mentioned this pull request Jan 31, 2024

[Security Solution] [Elastic AI Assistant] Adds internal Get Evaluate API and migrates Post Evaluate API to OAS #176025

Merged

2 tasks

CoenWarmer pushed a commit to CoenWarmer/kibana that referenced this pull request Feb 15, 2024

Revert "[Security Solution] [Elastic AI Assistant] Adds internal `Get…

9cbd119

… Evaluate` API and migrates `Post Evaluate` API to OAS (elastic#175338)" This reverts commit 38f0a7a.

fkanout pushed a commit to fkanout/kibana that referenced this pull request Mar 4, 2024

Revert "[Security Solution] [Elastic AI Assistant] Adds internal `Get…

e8086b8

… Evaluate` API and migrates `Post Evaluate` API to OAS (elastic#175338)" This reverts commit 38f0a7a.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Security Solution] [Elastic AI Assistant] Adds internal `Get Evaluate` API and migrates `Post Evaluate` API to OAS#175338

[Security Solution] [Elastic AI Assistant] Adds internal `Get Evaluate` API and migrates `Post Evaluate` API to OAS#175338
spong merged 7 commits intoelastic:mainfrom
spong:eval+++

spong commented Jan 23, 2024 •

edited

Loading

Uh oh!

elasticmachine commented Jan 23, 2024

Uh oh!

spong Jan 29, 2024

Uh oh!

kibana-ci commented Jan 29, 2024

API count

Uh oh!

stephmilovic left a comment

Uh oh!

jbudz commented Jan 31, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

spong commented Jan 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Checklist

Uh oh!

elasticmachine commented Jan 23, 2024

Uh oh!

spong Jan 29, 2024

Choose a reason for hiding this comment

Uh oh!

kibana-ci commented Jan 29, 2024

💚 Build Succeeded

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

API count

History

Uh oh!

stephmilovic left a comment

Choose a reason for hiding this comment

Uh oh!

jbudz commented Jan 31, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

spong commented Jan 23, 2024 •

edited

Loading