[EDR Workflows][Endpoint Exception move] Opt-in mechanism for per-policy Endpoint Exceptions#259598
Conversation
645adb3 to
0f4fc7d
Compare
|
buildkite test this |
szwarckonrad
left a comment
There was a problem hiding this comment.
Thanks for addressing my comments. Looks good, great work!
| const hasSuperuserRole = userRoles.includes('superuser'); | ||
| const hasAdminRole = userRoles.includes('admin'); | ||
|
|
||
| const hasSuperuserPrivileges = isServerless ? hasAdminRole : hasSuperuserRole; |
There was a problem hiding this comment.
OH... I missread. I think this new property should really be the one assigned to canWriteAdminData. I can do that in a separate PR if you don't want ot include it here... but also: maybe you should also be using canWriteAdminData instead of introducing a new authz privilege?
| EndpointGetActionsStatusRequestQueryInput, | ||
| EndpointGetActionsStatusResponse, | ||
| } from './endpoint/actions/status/status.gen'; | ||
| import type { GetEndpointExceptionsPerPolicyOptInResponse } from './endpoint/endpoint_exceptions_per_policy_opt_in/endpoint_exceptions_per_policy_opt_in.gen'; |
There was a problem hiding this comment.
I guess we're using the ZOD schemas generated from API docs now? I though we were sticking with Kibana Schemas for our APIs
There was a problem hiding this comment.
oh, yeah, i did not notice that the generated schemas are not used, although 27 of our API docs enabled x-codegen-enabled, and only 15 disabled it.
do we have a reason not to use these? if yes, i'll switch these to a hand-made schema, otherwise i think it is nice to have a single source of truth... especially if we have to write the API docs anyway
natasha-moore-elastic
left a comment
There was a problem hiding this comment.
Docs link LGTM
denar50
left a comment
There was a problem hiding this comment.
Code only review for files owned by the Detection Engine team. LGTM!
| </EuiToolTip> | ||
| <EuiSpacer size="l" /> | ||
| {!endpointExceptionsMovedUnderManagement ? ( | ||
| {endpointPerPolicyOptIn?.status === true ? ( |
There was a problem hiding this comment.
useGetEndpointExceptionsPerPolicyOptIn has a loading state. Should you take that on into account here?
There was a problem hiding this comment.
i think this should be fine: this way we use the GhostFormField (in order to not loose the data from the form) when loading is finished and opt-in is done, so we practically show the checkbox during loading - which shouldn't be disturbing as it is hidden in a closed 'advanced options' section anyway. but, in case fetching the opt-in state fails, we go back to the old behavior, i.e. showing the checkbox, which should be useful for now.
| title="moved" | ||
| /> | ||
| ); | ||
| } else if (endpointPerPolicyOptIn?.reason === 'userOptedIn') { |
There was a problem hiding this comment.
nit: this else can be removed.
💚 Build Succeeded
Metrics [docs]Module Count
Async chunks
Page load bundle
History
|
…icy Endpoint Exceptions (elastic#259598) ## Summary This PR adds an opt-in mechanism that allows using the so far global-only Endpoint Exceptions on a per-policy basis. > [!note] > Hidden behind feature flag (as part of the Endpoint exception move effort): > ``` > xpack.securitySolution.enableExperimental: > - endpointExceptionsMovedUnderManagement > ``` ### Behavior in short - new deployments are automatically opted in. they also won't see the 'Endpoint exceptions moved' and 'Endpoint exceptions not evaluated on rules anymore' callouts (from elastic#258556). - upgraded environments will see a new callout on the Endpoint exceptions page that suggests to enable per-policy behavior - superuser role (ESS) or admin role (serverless) users are able to perform the opt-in - other users will see a message to contact their administrators - after manually opting in - Endpoint exceptions can be assigned to package policies, like other endpoint artifacts - but, they won't be evaluated during rule execution anymore (see elastic#233289) - and they cannot be added to detection rules anymore (see elastic#242480) ### Data Opt-in status is stored in the `ReferenceDataClient`, and it's defaulted during plugin start phase either to: - `{ status: false }`, in case endpoint exception list exists (i.e. upgrade, it's created every time), - `{ status: true, reason: 'newDeployment' }`, in case the list does not exist. ### API 2 new API endpoints are added: - `GET /internal/api/endpoint/endpoint_exceptions_per_policy_opt_in` to receive the opt-in status on the UI, - `POST /internal/api/endpoint/endpoint_exceptions_per_policy_opt_in` to perform the opt-in, that sets the status to `{ status: true, reason: 'userOptedIn' }` ### UI #### empty page is... empty <img width="400" height="793" alt="image" src="https://github.com/user-attachments/assets/e7b72ab9-2a42-49ca-b2e9-2447e3ff9dd3" /> #### callout for non-superusers <img width="1280" height="516" alt="image" src="https://github.com/user-attachments/assets/f88a04a6-b8bb-46be-8f5f-ece5f1d89cf2" /> #### callout for superusers / admins <img width="1215" height="554" alt="image" src="https://github.com/user-attachments/assets/e2552d27-ac9d-4dfe-84da-48012fdf57fc" /> #### menu action for superusers / admins in case the callout is dismissed. although, it will come up again as it's only dismissed for the session. <img width="335" height="204" alt="image" src="https://github.com/user-attachments/assets/5d7a23e6-2428-4f8f-b941-0f97e8bd7750" /> #### modal <img width="925" height="564" alt="image" src="https://github.com/user-attachments/assets/0cbc4573-b052-4e1f-82c5-13298532f7fe" /> ### recording #### opt-in process and endpoint exceptions pages https://github.com/user-attachments/assets/de8204bd-72d2-4a99-b51c-b86771c577c3 ### hiding stuff behind opt-in status - endpoint exception related per-policy activities are available only after opt-in, like - per-policy assignment in the form, - assigning exceptions to policies in the Policy details page Endpoint exceptions tab, - `ManifestManager` also uses per-policy EE after opt-in only. - 'endpoint exceptions moved' informative callouts are shown without opting in (due to the FF enabled), or after _manual_ opt-in (i.e. are hidden on new deployments) - see the details in the previous PR: elastic#258556 - 'endpoint exceptions are not evaluated on rules' informative callouts are shown only after _manually_ opting in (i.e. it needs opt-in, but isn't shown on new deployments) - see the details in the previous PR: elastic#258556 ## Testing To test this, - either add some data (endpoint exceptions), and then turn on the feature flag `xpack.securitySolution.enableExperimental.endpointExceptionsMovedUnderManagement`, to simulate an 'upgrade' scenario, - or start with the feature flag turned on, to simulate a 'new deployment' scenario. ### Reading opt-in status dev console: ``` GET .kibana_security_solution/_doc/security:reference-data:ENDPOINT-EXCEPTIONS-PER-POLICY-OPT-IN-STATUS ``` ### Deleting opt-in status you can also delete the opt-in status, so you're back to the 'upgrade' scenario. for this, you need a `system_indices_superuser`, here's a quick console command to create one: ``` POST kbn://internal/security/users/testuser { "password": "changeme", "username": "testuser", "full_name": "", "email": "", "roles": [ "system_indices_superuser", "superuser" ] } ``` and then delete like this: ``` DELETE .kibana_security_solution/_doc/security:reference-data:ENDPOINT-EXCEPTIONS-PER-POLICY-OPT-IN-STATUS ``` ## todo docs ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
…icy Endpoint Exceptions (elastic#259598) ## Summary This PR adds an opt-in mechanism that allows using the so far global-only Endpoint Exceptions on a per-policy basis. > [!note] > Hidden behind feature flag (as part of the Endpoint exception move effort): > ``` > xpack.securitySolution.enableExperimental: > - endpointExceptionsMovedUnderManagement > ``` ### Behavior in short - new deployments are automatically opted in. they also won't see the 'Endpoint exceptions moved' and 'Endpoint exceptions not evaluated on rules anymore' callouts (from elastic#258556). - upgraded environments will see a new callout on the Endpoint exceptions page that suggests to enable per-policy behavior - superuser role (ESS) or admin role (serverless) users are able to perform the opt-in - other users will see a message to contact their administrators - after manually opting in - Endpoint exceptions can be assigned to package policies, like other endpoint artifacts - but, they won't be evaluated during rule execution anymore (see elastic#233289) - and they cannot be added to detection rules anymore (see elastic#242480) ### Data Opt-in status is stored in the `ReferenceDataClient`, and it's defaulted during plugin start phase either to: - `{ status: false }`, in case endpoint exception list exists (i.e. upgrade, it's created every time), - `{ status: true, reason: 'newDeployment' }`, in case the list does not exist. ### API 2 new API endpoints are added: - `GET /internal/api/endpoint/endpoint_exceptions_per_policy_opt_in` to receive the opt-in status on the UI, - `POST /internal/api/endpoint/endpoint_exceptions_per_policy_opt_in` to perform the opt-in, that sets the status to `{ status: true, reason: 'userOptedIn' }` ### UI #### empty page is... empty <img width="400" height="793" alt="image" src="https://github.com/user-attachments/assets/e7b72ab9-2a42-49ca-b2e9-2447e3ff9dd3" /> #### callout for non-superusers <img width="1280" height="516" alt="image" src="https://github.com/user-attachments/assets/f88a04a6-b8bb-46be-8f5f-ece5f1d89cf2" /> #### callout for superusers / admins <img width="1215" height="554" alt="image" src="https://github.com/user-attachments/assets/e2552d27-ac9d-4dfe-84da-48012fdf57fc" /> #### menu action for superusers / admins in case the callout is dismissed. although, it will come up again as it's only dismissed for the session. <img width="335" height="204" alt="image" src="https://github.com/user-attachments/assets/5d7a23e6-2428-4f8f-b941-0f97e8bd7750" /> #### modal <img width="925" height="564" alt="image" src="https://github.com/user-attachments/assets/0cbc4573-b052-4e1f-82c5-13298532f7fe" /> ### recording #### opt-in process and endpoint exceptions pages https://github.com/user-attachments/assets/de8204bd-72d2-4a99-b51c-b86771c577c3 ### hiding stuff behind opt-in status - endpoint exception related per-policy activities are available only after opt-in, like - per-policy assignment in the form, - assigning exceptions to policies in the Policy details page Endpoint exceptions tab, - `ManifestManager` also uses per-policy EE after opt-in only. - 'endpoint exceptions moved' informative callouts are shown without opting in (due to the FF enabled), or after _manual_ opt-in (i.e. are hidden on new deployments) - see the details in the previous PR: elastic#258556 - 'endpoint exceptions are not evaluated on rules' informative callouts are shown only after _manually_ opting in (i.e. it needs opt-in, but isn't shown on new deployments) - see the details in the previous PR: elastic#258556 ## Testing To test this, - either add some data (endpoint exceptions), and then turn on the feature flag `xpack.securitySolution.enableExperimental.endpointExceptionsMovedUnderManagement`, to simulate an 'upgrade' scenario, - or start with the feature flag turned on, to simulate a 'new deployment' scenario. ### Reading opt-in status dev console: ``` GET .kibana_security_solution/_doc/security:reference-data:ENDPOINT-EXCEPTIONS-PER-POLICY-OPT-IN-STATUS ``` ### Deleting opt-in status you can also delete the opt-in status, so you're back to the 'upgrade' scenario. for this, you need a `system_indices_superuser`, here's a quick console command to create one: ``` POST kbn://internal/security/users/testuser { "password": "changeme", "username": "testuser", "full_name": "", "email": "", "roles": [ "system_indices_superuser", "superuser" ] } ``` and then delete like this: ``` DELETE .kibana_security_solution/_doc/security:reference-data:ENDPOINT-EXCEPTIONS-PER-POLICY-OPT-IN-STATUS ``` ## todo docs ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
…m Alerts page (#262282) ## Summary > [!note] > After `endpointExceptionsMovedUnderManagement` FF is enabled, and after user [has opted in](#259598) to per-policy Endpoint exceptions... ...the Endpoint exception create form shows the Policy selector: <img width="573" height="99" alt="image" src="https://github.com/user-attachments/assets/a69b35b6-f55b-4e15-aa59-64f29eb6a64b" /> This should be defaulted to 'Global' when trying to create an Endpoint exception on the Alerts page: - to harmonize with historic behavior: so far Endpoint exceptions used to be always global, - to harmonize with Endpoint artifact pages: there we also default to global. ## Screen recording ### User with Global artifact management privilege https://github.com/user-attachments/assets/3058f0fd-7bea-4b2d-ab7f-f14912f8bd41 ### User without Global artifact management privilege https://github.com/user-attachments/assets/4f93e393-674b-462f-ba2c-8d4e31121983 > [!note] > actually this cannot be tested in this PR manually, because it is hidden behind another hidden bug: elastic/security-team#16784 > but the unit test shows the behavior ## Todo - [ ] update backport labels in case 9.4 branch is cut ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
…lag (#260983) > [!CAUTION] > Must be retargeted to 9.4 branch as soon as it's cut. Serverless release will follow later. ## Summary This PR enables the Security Solution feature flag `endpointExceptionsMovedUnderManagement` to: - hide Endpoint exceptions from Detections and Shared exception list pages, - instead, show Endpoint exceptions under Endpoint / Artifacts, - add an opt-in mechanism to allow users to opt-in to per-policy usage for Endpoint exceptions, - and add export/import functionality to all Endpoint artifacts And in order to do this, it: - adapts some of the tests, - deletes some obsolete ones, including fixtures, - removes Endpoint exception privilege condition for showing Shared exception list page (see [this comment](#239634 (comment))) - enables API documentation (see [this comment](#259598 (comment))) ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - --------- Co-authored-by: Konrad Szwarc <konrad.szwarc@elastic.co>
…ature flag (elastic#260983) > [!CAUTION] > Must be retargeted to 9.4 branch as soon as it's cut. Serverless release will follow later. This PR enables the Security Solution feature flag `endpointExceptionsMovedUnderManagement` to: - hide Endpoint exceptions from Detections and Shared exception list pages, - instead, show Endpoint exceptions under Endpoint / Artifacts, - add an opt-in mechanism to allow users to opt-in to per-policy usage for Endpoint exceptions, - and add export/import functionality to all Endpoint artifacts And in order to do this, it: - adapts some of the tests, - deletes some obsolete ones, including fixtures, - removes Endpoint exception privilege condition for showing Shared exception list page (see [this comment](elastic#239634 (comment))) - enables API documentation (see [this comment](elastic#259598 (comment))) Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - --------- Co-authored-by: Konrad Szwarc <konrad.szwarc@elastic.co>
Summary
This PR adds an opt-in mechanism that allows using the so far global-only Endpoint Exceptions on a per-policy basis.
Note
Hidden behind feature flag (as part of the Endpoint exception move effort):
Behavior in short
Data
Opt-in status is stored in the
ReferenceDataClient, and it's defaulted during plugin start phase either to:{ status: false }, in case endpoint exception list exists (i.e. upgrade, it's created every time),{ status: true, reason: 'newDeployment' }, in case the list does not exist.API
2 new API endpoints are added:
GET /internal/api/endpoint/endpoint_exceptions_per_policy_opt_into receive the opt-in status on the UI,POST /internal/api/endpoint/endpoint_exceptions_per_policy_opt_into perform the opt-in, that sets the status to{ status: true, reason: 'userOptedIn' }UI
empty page is... empty
callout for non-superusers
callout for superusers / admins
menu action for superusers / admins
in case the callout is dismissed. although, it will come up again as it's only dismissed for the session.
modal
recording
opt-in process and endpoint exceptions pages
Screen.Recording.2026-03-27.at.19.28.53.mov
hiding stuff behind opt-in status
ManifestManageralso uses per-policy EE after opt-in only.Testing
To test this,
xpack.securitySolution.enableExperimental.endpointExceptionsMovedUnderManagement, to simulate an 'upgrade' scenario,Reading opt-in status
dev console:
Deleting opt-in status
you can also delete the opt-in status, so you're back to the 'upgrade' scenario.
for this, you need a
system_indices_superuser, here's a quick console command to create one:and then delete like this:
todo
docs
Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.