[Security solution][Endpoint] Add Host Isolation related data to the endpoint generator and test data loader by paul-tavares · Pull Request #100727 · elastic/kibana

paul-tavares · 2021-05-26T19:29:24Z

Summary

new data generator for both Isolate and UnIsolate Fleet actions
Endpoint test/dev data loader was enhanced to load actions for each endpoint (aka: agent) when run with the --fleet option

Load Data

node x-pack/plugins/security_solution/scripts/endpoint/resolver_generator.js --auth elastic:changeme --delete --numHosts=10 --numDocs=2 --fleet

After running the above, Actions and Action Responses would be added for each host:

GET .fleet-actions/_search

[
    {
      "_index" : ".fleet-actions-7",
      "_id" : "dc869626-107b-43ff-b079-fbb6ae243951",
      "_score" : 1.0,
      "_source" : {
        "@timestamp" : "2021-05-24T13:11:17.625Z",
        "expiration" : "2021-06-23T13:11:17.625Z",
        "agents" : [
          "01c42a3f-5935-40ef-992d-f27434910427"
        ],
        "action_id" : "dc869626-107b-43ff-b079-fbb6ae243951",
        "type" : "POLICY_REASSIGN"
      }
    },
    {
      "_index" : ".fleet-actions-7",
      "_id" : "RcHdnnkBA_1gpn-NujXN",
      "_score" : 1.0,
      "_source" : {
        "action_id" : "1a11fcb3-ef4c-48bf-a3a8-5f57e85b4a95",
        "@timestamp" : "2021-05-24T14:53:21.996Z",
        "expiration" : "2021-06-07T14:53:21.996Z",
        "type" : "INPUT_ACTION",
        "input_type" : "endpoint",
        "agents" : [
          "a90b5ff6-9a9d-4a2c-863d-724587cbef48"
        ],
        "user_id" : "elastic",
        "data" : {
          "command" : "isolate",
          "comment" : ""
        }
      }
    },
    {
      "_index" : ".fleet-actions-7",
      "_id" : "RsHdnnkBA_1gpn-N1TVC",
      "_score" : 1.0,
      "_source" : {
        "action_id" : "3305c14e-7262-4018-9082-5c2b0d09e555",
        "@timestamp" : "2021-05-24T14:53:28.769Z",
        "expiration" : "2021-06-07T14:53:28.769Z",
        "type" : "INPUT_ACTION",
        "input_type" : "endpoint",
        "agents" : [
          "a90b5ff6-9a9d-4a2c-863d-724587cbef48"
        ],
        "user_id" : "elastic",
        "data" : {
          "command" : "isolate",
          "comment" : ""
        }
      }
    }
]

…-host-isolation-to-generator

elasticmachine · 2021-05-26T19:29:27Z

Pinging @elastic/security-onboarding-and-lifecycle-mgt (Team:Onboarding and Lifecycle Mgt)

paul-tavares · 2021-05-26T19:30:40Z

x-pack/plugins/security_solution/common/endpoint/types/actions.ts

  };
 }

+export interface EndpointActionResponse {


Does this already exist anywhere else? @ashokaditya maybe in your PR?

Might exist in Ash's draft. Does not exist anywhere else yet, since we don't really know what the success structure looks like (or even failure when it comes from the endpoint).

yeah, I was wondering that as well. Feels like the Agent should know what an error/success looks like since they are the ones writing this to es. (just by looking at the schema, I thought that if error was populated, then it "failed", else, it was "success" 🤷‍♂️ )

Agent knows about the fields it writes. It does not know about the fields Endpoint writes (and vice versa).

AFAIK it treats the response given via endpoint as a total unparsed black box, wraps it with its own fields, and then passes to fleet server (where maybe more fields are written? no? who knows?).

that error field is coming from either fleet-server or agent who have a failure in their system before being able to deliver to endpoint.

pzl

looks good. had thoughts, but nothing big

x-pack/plugins/security_solution/common/endpoint/data_generators/base_data_generator.ts

x-pack/plugins/security_solution/common/endpoint/generate_data.ts

x-pack/plugins/security_solution/common/endpoint/index_data.ts

pzl · 2021-05-26T19:49:45Z

x-pack/plugins/security_solution/common/endpoint/types/actions.ts

  };
 }

+export interface EndpointActionResponse {


Might exist in Ash's draft. Does not exist anywhere else yet, since we don't really know what the success structure looks like (or even failure when it comes from the endpoint).

pzl · 2021-05-26T20:57:05Z

x-pack/plugins/security_solution/common/endpoint/generate_data.ts

        },
        state: {
-          isolation: false,
+          isolation: isIsolated,


future improvement: Math.random() < 0.2 ? !isIsolated : isIsolated perhaps

…-host-isolation-to-generator

kibanamachine · 2021-05-27T14:53:05Z

💚 Build Succeeded

Metrics [docs]

Unknown metric groups

References to deprecated APIs

id	before	after	diff
`ml`	121	115	-6
`securitySolution`	386	342	-44
total			-50

History

💔 Build #127529 failed 0ffdafe

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @paul-tavares

…endpoint generator and test data loader (elastic#100727) * Generate random isolation values for endpoint metadata * Generator for Fleet Actions * Added creation of actions to the index test data loader

kibanamachine · 2021-05-27T15:58:34Z

💚 Backport successful

Status	Branch	Result
✅	7.x

This backport PR will be merged automatically after passing CI.

… to the endpoint generator and test data loader (#100727)" This reverts commit 57f59bd.

jbudz · 2021-05-27T17:23:24Z

This was reverted by 5dde07f.

https://kibana-ci.elastic.co/job/elastic+kibana+master/14367/execution/node/449/log/

…endpoint generator and test data loader (elastic#100727) * Generate random isolation values for endpoint metadata * Generator for Fleet Actions * Added creation of actions to the index test data loader (cherry picked from commit 57f59bd)

…ort for Host Isolation (#100813) Re-introduces the changes from #100727 which was backed out due to a bug. Changes included: * Generate random isolation values for endpoint metadata * Generator for Fleet Actions * Added creation of actions to the index test data loader Plus: * Fix generator `randomBoolean()` to ensure it works with seeded random numbers * Update resolver snapshots due to additional call to randomizer

…ort for Host Isolation (elastic#100813) Re-introduces the changes from elastic#100727 which was backed out due to a bug. Changes included: * Generate random isolation values for endpoint metadata * Generator for Fleet Actions * Added creation of actions to the index test data loader Plus: * Fix generator `randomBoolean()` to ensure it works with seeded random numbers * Update resolver snapshots due to additional call to randomizer

…ort for Host Isolation (#100813) (#100904) Re-introduces the changes from #100727 which was backed out due to a bug. Changes included: * Generate random isolation values for endpoint metadata * Generator for Fleet Actions * Added creation of actions to the index test data loader Plus: * Fix generator `randomBoolean()` to ensure it works with seeded random numbers * Update resolver snapshots due to additional call to randomizer Co-authored-by: Paul Tavares <56442535+paul-tavares@users.noreply.github.com>

paul-tavares added 5 commits May 25, 2021 15:13

Generate random isolation values for endpoint metadata

d19aeae

Generator for Fleet Actions

4d9c717

Added creation of actions to the index data generator

0d4e75d

randomize data.command value

4d45c0a

Merge remote-tracking branch 'upstream/master' into task/olm-1117-add…

fd57e28

…-host-isolation-to-generator

paul-tavares added v8.0.0 release_note:skip Skip the PR/issue when compiling release notes Team:Defend Workflows “EDR Workflows” sub-team of Security Solution v7.14.0 auto-backport Deprecated - use backport:version if exact versions are needed labels May 26, 2021

paul-tavares requested a review from a team as a code owner May 26, 2021 19:29

paul-tavares self-assigned this May 26, 2021

paul-tavares commented May 26, 2021

View reviewed changes

pzl approved these changes May 26, 2021

View reviewed changes

adjustments from PR review

0ffdafe

pzl reviewed May 26, 2021

View reviewed changes

paul-tavares added 2 commits May 27, 2021 08:37

Adjust .randomBoolean()

9e8bd54

Merge remote-tracking branch 'upstream/master' into task/olm-1117-add…

d48ae6b

…-host-isolation-to-generator

paul-tavares merged commit 57f59bd into elastic:master May 27, 2021

paul-tavares deleted the task/olm-1117-add-host-isolation-to-generator branch May 27, 2021 15:55

kibanamachine mentioned this pull request May 27, 2021

[Security solution][Endpoint] Add Host Isolation related data to the endpoint generator and test data loader (#100727) #100796

Closed

jbudz added a commit that referenced this pull request May 27, 2021

Revert "[Security solution][Endpoint] Add Host Isolation related data…

5dde07f

… to the endpoint generator and test data loader (#100727)" This reverts commit 57f59bd.

paul-tavares mentioned this pull request May 27, 2021

[Security Solution][Endpoint] Endpoint generator and data loader support for Host Isolation #100813

Merged

Conversation

paul-tavares commented May 26, 2021

Summary

Uh oh!

elasticmachine commented May 26, 2021

Uh oh!

paul-tavares May 26, 2021

Choose a reason for hiding this comment

Uh oh!

pzl May 26, 2021

Choose a reason for hiding this comment

Uh oh!

paul-tavares May 26, 2021

Choose a reason for hiding this comment

Uh oh!

pzl May 26, 2021

Choose a reason for hiding this comment

Uh oh!

pzl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pzl May 26, 2021

Choose a reason for hiding this comment

Uh oh!

pzl May 26, 2021

Choose a reason for hiding this comment

Uh oh!

kibanamachine commented May 27, 2021

💚 Build Succeeded

Metrics [docs]

References to deprecated APIs

History

Uh oh!

kibanamachine commented May 27, 2021

💚 Backport successful

Uh oh!

jbudz commented May 27, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants