Skip to content

[EDR Workflows] Update fetchAgentPolicyInfo query in CrowdStrike#234772

Merged
tomsonpl merged 5 commits intoelastic:mainfrom
tomsonpl:fix-cs
Sep 17, 2025
Merged

[EDR Workflows] Update fetchAgentPolicyInfo query in CrowdStrike#234772
tomsonpl merged 5 commits intoelastic:mainfrom
tomsonpl:fix-cs

Conversation

@tomsonpl
Copy link
Contributor

@tomsonpl tomsonpl commented Sep 11, 2025

We identified that some event's coming from CrowdStrike were missing event.created field, thus making using response actions impossible. This PR fixes that:

Why [at]timestamp is Safer

  1. Field Availability: [at]timestamp is a standard Elasticsearch field that should be present in virtually all documents indexed into Elasticsearch.
    It's automatically set by Elasticsearch if not provided by the document.
  2. ECS Inconsistency: event.created is an ECS field that may not always be populated by all data sources.
    field populated.
  3. Integration-Specific Behavior: Third-party EDR integrations (like CrowdStrike) may not consistently populate ECS fields like
    event.created, but they should always have @timestamp since it's fundamental to Elasticsearch document lifecycle.

@tomsonpl tomsonpl self-assigned this Sep 11, 2025
@tomsonpl tomsonpl added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting Team:Defend Workflows “EDR Workflows” sub-team of Security Solution v9.2.0 v9.1.5 labels Sep 11, 2025
@tomsonpl
Copy link
Contributor Author

/ci

@tomsonpl tomsonpl marked this pull request as ready for review September 12, 2025 08:15
@tomsonpl tomsonpl requested a review from a team as a code owner September 12, 2025 08:15
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-defend-workflows (Team:Defend Workflows)

@tomsonpl tomsonpl requested review from ashokaditya and removed request for szwarckonrad September 12, 2025 08:16
@ashokaditya
Copy link
Member

Since @timestamp is always present with the crowdstrike ingested alert data, @paul-tavares do you know why we chose event.created field that may or may not be present, in the first place? Looks like it was introduced as a part of space awareness changes (#218175) in https://github.com/elastic/kibana/pull/218175/files#diff-ddd62b8dcd4453439ec6640ec7ef9fb1e76572072ec2bc838e86265237f3b2a1R124

I am wondering if this bug was introduced in 9.1, and I'm concerned if changing this would break CS response actions.

@tomsonpl tomsonpl requested review from paul-tavares and removed request for gergoabraham September 16, 2025 08:24
size: 1,
_source: ['agent', 'device.id', 'event.created'],
sort: [{ 'event.created': 'desc' }],
_source: ['agent', 'device.id', '@timestamp'],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The goal of this code, that was introduced in 9.1 in support of spaces, is to get the most recent information about the host. I used event.created because my understanding was that was the most appropriate field. If using @timestamp makes more sense for CrowdStrike, then I don't see an issue here.

What is interesting to me is: why/how are CrowdStrike documents being ingested into ES and missing this field? (but that's just me)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a valid question — I was actually looking into it, and my guess is that it may be omitted if timestamp and event.created are identical — but that's just my guess.

https://github.com/elastic/ecs/blob/main/schemas/event.yml#L766

@elasticmachine
Copy link
Contributor

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #1 / discover/security/context_awareness cell renderer ES|QL mode should render alert workflow status badge

Metrics [docs]

Saved Objects .kibana field count

Every field in each saved object type adds overhead to Elasticsearch. Kibana needs to keep the total field count below Elasticsearch's default limit of 1000 fields. Only specify field mappings for the fields you wish to search on or query. See https://www.elastic.co/guide/en/kibana/master/saved-objects-service.html#_mappings

id before after diff
_data_stream_timestamp 1 - -1
_doc_count 1 - -1
_ignored_source 1 - -1
_index_mode 1 - -1
_inference_fields 1 - -1
_tier 1 - -1
apm-custom-dashboards 5 - -5
apm-server-schema 2 - -2
apm-service-group 5 - -5
application_usage_daily 2 - -2
config 2 - -2
config-global 2 - -2
coreMigrationVersion 1 - -1
created_at 1 - -1
created_by 1 - -1
entity-definition 9 - -9
entity-discovery-api-key 2 - -2
event_loop_delays_daily 2 - -2
favorites 4 - -4
file 11 - -11
file-upload-usage-collection-telemetry 3 - -3
fileShare 5 - -5
infra-custom-dashboards 4 - -4
infrastructure-monitoring-log-view 2 - -2
intercept_trigger_record 5 - -5
legacy-url-alias 7 - -7
managed 1 - -1
ml-job 6 - -6
ml-module 13 - -13
ml-trained-model 7 - -7
monitoring-telemetry 2 - -2
namespace 1 - -1
namespaces 1 - -1
observability-onboarding-state 2 - -2
originId 1 - -1
product-doc-install-status 7 - -7
references 4 - -4
sample-data-telemetry 3 - -3
security-ai-prompt 8 - -8
slo 11 - -11
space 5 - -5
synthetics-monitor 34 - -34
synthetics-monitor-multi-space 34 - -34
tag 4 - -4
type 1 - -1
typeMigrationVersion 1 - -1
ui-metric 2 - -2
updated_at 1 - -1
updated_by 1 - -1
upgrade-assistant-ml-upgrade-operation 3 - -3
upgrade-assistant-reindex-operation 3 - -3
uptime-synthetics-api-key 2 - -2
url 5 - -5
usage-counters 2 - -2
total -246

History

cc @tomsonpl

@tomsonpl tomsonpl merged commit a7d04fb into elastic:main Sep 17, 2025
12 checks passed
@tomsonpl tomsonpl added backport:version Backport to applied version labels and removed backport:skip This PR does not require backporting labels Sep 17, 2025
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 9.1

https://github.com/elastic/kibana/actions/runs/17799801972

@kibanamachine
Copy link
Contributor

Starting backport for target branches: 9.1

https://github.com/elastic/kibana/actions/runs/17799801788

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Sep 17, 2025
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Sep 17, 2025
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
9.1

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

1 similar comment
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
9.1

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Sep 17, 2025
#234772) (#235373)

# Backport

This will backport the following commits from `main` to `9.1`:
- [[EDR Workflows] Update fetchAgentPolicyInfo query in CrowdStrike
(#234772)](#234772)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Tomasz
Ciecierski","email":"tomasz.ciecierski@elastic.co"},"sourceCommit":{"committedDate":"2025-09-17T13:47:57Z","message":"[EDR
Workflows] Update fetchAgentPolicyInfo query in CrowdStrike
(#234772)","sha":"a7d04fb7c762fa36e0c11f2f5ae33315678093a0","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Defend
Workflows","backport:version","v9.2.0","v9.1.5"],"title":"[EDR
Workflows] Update fetchAgentPolicyInfo query in
CrowdStrike","number":234772,"url":"https://github.com/elastic/kibana/pull/234772","mergeCommit":{"message":"[EDR
Workflows] Update fetchAgentPolicyInfo query in CrowdStrike
(#234772)","sha":"a7d04fb7c762fa36e0c11f2f5ae33315678093a0"}},"sourceBranch":"main","suggestedTargetBranches":["9.1"],"targetPullRequestStates":[{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/234772","number":234772,"mergeCommit":{"message":"[EDR
Workflows] Update fetchAgentPolicyInfo query in CrowdStrike
(#234772)","sha":"a7d04fb7c762fa36e0c11f2f5ae33315678093a0"}},{"branch":"9.1","label":"v9.1.5","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Tomasz Ciecierski <tomasz.ciecierski@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:version Backport to applied version labels release_note:skip Skip the PR/issue when compiling release notes Team:Defend Workflows “EDR Workflows” sub-team of Security Solution v9.1.4 v9.1.5 v9.2.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants