[Security solution] Endpoint exception RBAC in ESS siemV4 for 9.2

[gergoabraham]

should be merged on the same week as:

• [EDR Workflows] Fix endpoint exception RBAC UI issues #235119

Tip

Apologies for the huge PR. To help getting started, as we're getting closer to feature freeze:

• The meaningful changes are only ca. 600 lines, while one snapshot test update (authorization.ts) is 3k lines on its own, and the rest is test updates and the necessary small updates in different plugins. But all the changes belong together unfortunately.
• Exploratory migration tests are already performed from siemV3 to siemV4, both on ESS and serverless environments, both with custom and pre-defined roles when applicable. (result 1, result 2, result 3)
• I added some comments on the changes with this Tip format to help the review.

Summary

The goal of this PR is to add Endpoint Exceptions sub-feature privilege to ESS and Serverless in the same way.

(with new text)

ESS

On ESS, Endpoint Exceptions did not have its own sub-privilege: it was included implicitly in the Security privilege (Security:ALL => EE:ALL, Security:READ => EE:READ). Therefore, a replacedBy role migration and api backward compatibility was added, in order to preserve behavior for the users.

Serverless

On Serverless, Endpoint Exceptions sub-feature privilege has already existed in the following way:

• with Security sub-feature privilege customization disabled, it was included in Security privilege (not like other artifact sub-privileges as they are never included).
• with sub-feature privilege customization enabled, it was defined by the Endpoint Exception sub-feature privilege manually.

Here, a replacedBy role migration and api backward compatibility was also added.

Including Endpoint Exceptions in Security:READ/ALL

Endpoint Exceptions is not included anymore in Security:READ/ALL privileges, it always has to be explicitly allowed, similarly to other artifacts.

Before (screenshot from serverless. on ESS it was hidden)	After

Feature deprecations / role migrations

security:NONE

flowchart LR

    subgraph siemV3[siem/siemV2/siemV3]
        none1[none]
    end

    subgraph siemV4
        none2[none]
    end

    none1 --> none2

Loading

security:READ - same for ESS and serverless

flowchart LR

    subgraph siemV2[siem/siemV2/siemV3]
        read1[read]
    end

    subgraph siemV4
        read2[minimal_read]
        endpoint_exceptions_read
    end

    read1 --enable sub-feature privilege customization toggle--> read2
    read1 --> endpoint_exceptions_read

Loading

security:MINIMAL_READ (when the sub-feature customizsation toggle is enabled)

ESS

flowchart LR

    subgraph siemV2[siem/siemV2/siemV3]
        read1[minimal_read]
    end

    subgraph siemV4
        read2[minimal_read]
        endpoint_exceptions_read
    end

    read1 --> read2
    read1 --> endpoint_exceptions_read

Loading

Serverless - no change

flowchart LR

    subgraph siemV2[siem/siemV2/siemV3]
        read1[minimal_read]
        eer1[endpoint_exceptions_read]
        eew1[endpoint_exceptions_all]
    end

    subgraph siemV4
        read2[minimal_read]
        eer2[endpoint_exceptions_read]
        eew2[endpoint_exceptions_all]
    end

    read1 --> read2
    eer1 --> eer2
    eew1 --> eew2

Loading

security:ALL - same for ESS and serverless

flowchart LR

    subgraph siemV2[siem/siemV2/siemV3]
        all1[all]
    end

    subgraph siemV4
        read2[minimal_all]
        endpoint_exceptions_all
    end

    all1 --enable sub-feature privilege customization toggle--> read2
    all1 --> endpoint_exceptions_all

Loading

security:MINIMAL_ALL (when the sub-feature customization toggle is enabled)

ESS

flowchart LR

    subgraph siemV2[siem/siemV2/siemV3]
        all1[minimal_all]
    end

    subgraph siemV4
        all2[minimal_all]
        endpoint_exceptions_all
    end

    all1 --> all2
    all1 --> endpoint_exceptions_all

Loading

Serverless - no change

flowchart LR

    subgraph siemV2[siem/siemV2/siemV3]
        all1[minimal_all]
        eer1[endpoint_exceptions_read]
        eew1[endpoint_exceptions_all]
    end

    subgraph siemV4
        all2[minimal_all]
        eer2[endpoint_exceptions_read]
        eew2[endpoint_exceptions_all]
    end

    all1 --> all2
    eer1 --> eer2
    eew1 --> eew2

Loading

Testing ✅ done

see result 1, result 2, result 3

It should be tested on cloud deploy, both serverless and ESS.

What to test?

• [serverless only] Pre-defined roles (like viewer, t1_analyst etc.) behave the same as before regarding endpoint exception access.
• [ESS] Users having security none/read/all privilege on previous siem versions have the same endpoint exception access as before.*
• [serverless+ESS] Deprecated roles are replaced as expected.*

* For these, custom roles can be created in the Dev console with previous siem versions.

Dev console commands

// Creating or updating `testrole` custom role on ESS and Serverless
PUT kbn://api/security/role/testrole
{
  "elasticsearch": {
    "cluster": [
      "manage"
    ],
    "indices": [
      {
        "names": [
          "logs-*",
          ".lists-*",
          ".items-*",
          ".alerts-security.alerts-*",
          ".siem-signals*"
        ],
        "privileges": [
          "read",
          "write",
          "view_index_metadata",
          "manage"
        ],
        "field_security": {
          "grant": [
            "*"
          ],
          "except": []
        },
        "allow_restricted_indices": false
      }
    ],
    "run_as": []
  },
  "kibana": [
    {
      "base": [],
      "feature": {
        "siem": [ // this can be siem | siemV2 | siemV3 | siemV4
          "all", // read | minimal_read | all | minimal_all, or without this
          "endpoint_exceptions_read", // only on serverless, _read or _all, or without this
          "global_artifact_management_all" // coming from siem|siemV2 this is added automatically, but siemV3|siemV4 this is needed for write access
        ],
        "dev_tools": [
          "all"
        ]
      },
      "spaces": [
        "*"
      ]
    }
  ]
}

// Reading the custom role as how it is stored
GET kbn://api/security/role/testrole

// Reading the custom role with replacing the deprecated features
GET kbn://api/security/role/testrole?replaceDeprecatedPrivileges=true

// Creating a user on ESS
POST kbn://internal/security/users/testuser
{
  "password": "changeme",
  "username": "testuser",
  "full_name": "",
  "email": "",
  "roles": [
    "testrole"
  ]
}

Note

Coming from siem or siemV2 security:ALL will include global_artifact_management_all as well. See #219566
Coming from siem will add new features next to siemV4: securitySolutionTimeline and securitySolutionNotes. See #201780

What not to test?

What should be seen is that the role migration works well. What won't be seen is that setting Endpoint Exception sub-privilege has the expected effect, as there are some authorization bugs on UI side. These bugs will being fixed while this PR is under review, in a follow-up PR. Issue: https://github.com/elastic/security-team/issues/13921

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

Release notes

Endpoint exceptions sub-privilege is added to ESS. From now on, Endpoint Exception access needs to be explicitly set both on ESS and Serverless for user roles.

Doc issue

• [Internal]: Endpoint Exceptions sub-feature privilege across offers docs-content#2974

[logeekal]

PR looks LGTM from investigations perspective. Thanks for detailed PR description. It helped in better understanding and testing.

[vgomez-el]

LGTM!

[jloleysens]

Focused my review on serverless.security.yml. Based on the description for siemv4 this makes sense.

[dasansol92]

@elasticmachine merge upstream

[dasansol92]

@elasticmachine merge upstream

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: fc313be
• Storybooks Preview
• Cloud Deployment
• Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-233433-fc313be25261
• Security Deployment

Failed CI Steps

• FTR Configs #112
• Defend Workflows Cypress Tests #9

Test Failures

• [job] [logs] FTR Configs #112 / alerting api integration security and spaces enabled - Group 2 Connectors webhook action OAuth2 client credentials should refresh the token once the previous one has expired

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
securitySolution	10.8MB	10.8MB	-34.0B

Unknown metric groups

References to deprecated APIs

id	before	after	diff
@kbn/test-suites-xpack-security	33	29	-4

History

• 💔 Build #344006 failed b570786
• 💔 Build #343974 failed 3b4e8bd
• 💔 Build #343862 failed 543ac6e
• 💚 Build #342633 succeeded f71227e

cc @gergoabraham