[Entity Analytics] Update entity maintainers to use new relationship schema (ids object) by seanrathier · Pull Request #262911 · elastic/kibana

seanrathier · 2026-04-13T21:35:27Z

Summary

Updates the communicates_with and accesses entity maintainers to write relationship values as { ids: string[] } objects instead of plain string arrays, aligning with the EntityRelationship schema introduced in #262242.

Both maintainers now carry { ids: string[] } end-to-end through their internal pipelines — from ProcessedEntityRecord through postprocessEsqlResults to buildEntityDoc/updateEntityRelationships.

Changes

communicates_with maintainer

types.ts — ProcessedEntityRecord.communicates_with: string[] → { ids: string[] }
postprocess_records.ts — wraps ESQL output in { ids: [...] }
run_maintainer.ts — logging updated to use .ids
update_entities.ts — mergeRecordsByEntityId reads .ids for iteration/deduplication

accesses maintainer

types.ts — ProcessedEntityRecord.accesses_frequently/infrequently: string[] → { ids: string[] }
postprocess_records.ts — wraps ESQL output in { ids: [...] } (+ new test file)
run_maintainer.ts — logging updated to use .ids
update_entities.ts — guards updated to .ids.length > 0 (+ new test file)

How to test

Run node scripts/jest x-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/maintainers --no-coverage — all 207 tests pass
Run node scripts/type_check --project x-pack/solutions/security/plugins/security_solution/tsconfig.json — no errors
Trigger a maintainer run and verify entity store documents contain { "ids": [...] } under relationship fields

Tracks the changes needed in communicates_with and accesses maintainers to align with the EntityRelationship schema introduced in elastic#262242. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ema update Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ids schema

…nship ids object

…ects for accesses relationships

…w ids schema

…s schema Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Update ProcessedEntityRecord usage in run_maintainer.test.ts to use the { ids: string[] } object format matching the updated type definition, and apply eslint auto-fix formatting to postprocess_records.test.ts. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

… to { ids: string[] } schema Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

elasticmachine · 2026-04-13T21:35:32Z

Pinging @elastic/siem (Team:SIEM)

elasticmachine · 2026-04-13T21:35:33Z

Pinging @elastic/siem (Team:SIEM)

elasticmachine · 2026-04-13T21:43:23Z

Pinging @elastic/contextual-security-apps (Team:Cloud Security)

elasticmachine · 2026-04-13T21:43:23Z

Pinging @elastic/security-entity-analytics (Team:Entity Analytics)

macroscopeapp · 2026-04-13T21:44:02Z

Approvability

Verdict: Needs human review

Schema change wrapping relationship arrays in { ids: [...] } objects affects runtime data shapes for entity analytics. Author does not own any of the 15 changed files (all owned by @elastic/security-entity-analytics), and two open review comments suggest code improvements. Designated code owners should review this schema migration.

^{You can customize Macroscope's approvability policy. Learn more.}

…rite

…cesses

… write Fixes elastic#262869 When multiple integrations produce ProcessedEntityRecord objects with the same entityId, the accesses maintainer previously mapped them 1:1 to bulk update objects. Last-writer-wins meant only one integration's host IDs survived. Add mergeRecordsByEntityId to accumulate host IDs across all records for the same entity before writing. The precedence rule ensures that if a host appears in accesses_frequently from any integration it is promoted out of accesses_infrequently.

… in accesses update

…262893

…ntainers-relationship-schema-262893

tiansivive

Good stuff, just a few suggestions

tiansivive · 2026-04-15T14:43:38Z

+  }
+
+  return merged;
 }


This works, but I found it harder to follow than it needs to be because the logic is spread across nested loops, branching, and multiple mutation phases. As a reader, I have to mentally track state changes while also figuring out the actual business rules.
I think this would be much cleaner if it were structured more linearly, with explicit steps like filtering valid records, grouping by entityId, merging ids, then applying the precedence rule.

It would make the intent much easier to read at a glance.

This is a bit nicer, and the comments do help, but I think the main readability issue is still the structure rather than the lack of explanation. It still relies on branching, nested iteration, and mutation across multiple phases, so I still have to trace the control flow pretty closely to understand it.
I was hoping for something where the steps are more directly reflected in the shape of the code, instead of needing comments to explain the flow.
Extracting to small named utility functions and then doing one chain of array methods like .map/.filter/.reduce or a similarly styled loop would help a lot

I took a bit more time on this, I think this should satisfy your concerns.
I tried to find more common ground between the maintainers for a shared function but it was minimal.

This code will likely no longer exist in 9.5

- Restructure mergeRecordsByEntityId into explicit linear steps (filter, group, merge, precedence) to improve readability - Replace `as Entity` coercion with `satisfies Entity` for stronger type safety Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

CAWilson94 · 2026-04-16T10:02:38Z

+        for (const t of r.communicates_with.ids) existing.targets.add(t);
      } else {
        merged.set(r.entityId, {
          entityType: r.entityType,


Nit: Not directly part of this PR, but I noticed an inconsistency between the two update_entities files:

In communicates_with/update_entities, entityType is typed as string (from MergedEntity)

In accesses/update_entities, EntityType is the Zod-derived enum from @kbn/entity-store/common

Unless MergedEntity.entityType intentionally needs to accept unknown values from raw data, it would be more consistent for both to use the EntityType enum, and runtime validation too.

Thanks for noticing that! I didn't have an agent review pass on the last commit.

CAWilson94

Desk Tested - working as described. Added one nit, otherwise LGTM with changes added 🚀

Threads EntityType (the enum) through ProcessedEntityRecord.entityType, postProcessEsqlResults, and MergedEntity instead of widening to string. Removes the as BulkObject['type'] coercion and switches as Entity to satisfies Entity for consistency with the accesses maintainer. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…262893

…e steps Extract a shared generic groupByEntityId utility and restructure both accesses and communicates_with update_entities into explicit named functions (filterValid / seed / merge / applyPrecedence) composed in a clear pipeline, removing the need for inline comments to explain control flow. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

tiansivive

Thanks for addressing everything

elasticmachine · 2026-04-16T20:35:45Z

⏳ Build in-progress

History

💚 Build #429208 succeeded 2a0392c
💔 Build #428280 failed c3e2829
💚 Build #426996 succeeded d0d0eae
💚 Build #426974 succeeded c752e35

cc @seanrathier

kibanamachine · 2026-04-16T21:28:24Z

Starting backport for target branches: 9.4

https://github.com/elastic/kibana/actions/runs/24534917239

…schema (ids object) (elastic#262911) (cherry picked from commit 33ee38a)

kibanamachine · 2026-04-16T21:37:25Z

💚 All backports created successfully

Status	Branch	Result
✅	9.4

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

…nship schema (ids object) (#262911) (#263917) # Backport This will backport the following commits from `main` to `9.4`: - [[Entity Analytics] Update entity maintainers to use new relationship schema (ids object) (#262911)](#262911)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  Co-authored-by: seanrathier <sean.rathier@gmail.com>

seanrathier and others added 10 commits April 13, 2026 16:13

docs: add design spec for entity maintainers relationship schema update

615f247

Tracks the changes needed in communicates_with and accesses maintainers to align with the EntityRelationship schema introduced in elastic#262242. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

docs: add implementation plan for entity maintainers relationship sch…

c563696

…ema update Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

feat(entity-analytics): update communicates_with relationship to new …

dc49cba

…ids schema

feat(entity-analytics): update ProcessedEntityRecord to carry relatio…

803ccb2

…nship ids object

feat(entity-analytics): update postprocessEsqlResults to emit ids obj…

a577916

…ects for accesses relationships

feat(entity-analytics): update accesses run_maintainer logging for ne…

8c4d0c1

…w ids schema

feat(entity-analytics): update accesses update_entities to use new id…

c6da530

…s schema Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

feat(entity-analytics): align communicates_with ProcessedEntityRecord…

0deeb0c

… to { ids: string[] } schema Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

lint fix

61d48bc

seanrathier requested a review from a team as a code owner April 13, 2026 21:35

seanrathier added Team:SIEM release_note:skip Skip the PR/issue when compiling release notes labels Apr 13, 2026

seanrathier requested a review from CAWilson94 April 13, 2026 21:35

seanrathier added Team:SIEM release_note:skip Skip the PR/issue when compiling release notes labels Apr 13, 2026

seanrathier added 2 commits April 13, 2026 17:41

revert: remove docs/superpowers from repo

c752e35

revert: remove docs/superpowers from repo

618f27f

seanrathier added Team:Cloud Security Cloud Security team related Team:Entity Analytics Security Entity Analytics Team and removed Team:SIEM labels Apr 13, 2026

seanrathier self-assigned this Apr 13, 2026

seanrathier added backport:version Backport to applied version labels v9.4.0 labels Apr 13, 2026

seanrathier added the ci:cloud-deploy Create or update a Cloud deployment label Apr 13, 2026

seanrathier enabled auto-merge (squash) April 13, 2026 21:45

seanrathier added 6 commits April 13, 2026 18:31

test(entity-analytics): add failing tests for accesses merge-before-w…

68374cb

…rite

test(entity-analytics): add deduplication and entityType tests for ac…

d9362ac

…cesses

refactor(entity-analytics): inline entity doc and add empty-ids guard…

d0d0eae

… in accesses update

Merge branch 'main' of github.com:elastic/kibana

ee31fa8

Merge branch 'main' into feat/entity-maintainers-relationship-schema-…

552051c

…262893

seanrathier disabled auto-merge April 14, 2026 23:46

Merge branch 'main' of github.com:elastic/kibana into feat/entity-mai…

2ae50e1

…ntainers-relationship-schema-262893

seanrathier removed the request for review from CAWilson94 April 15, 2026 14:35

tiansivive requested changes Apr 15, 2026

View reviewed changes

seanrathier requested a review from tiansivive April 15, 2026 15:41

CAWilson94 reviewed Apr 16, 2026

View reviewed changes

CAWilson94 approved these changes Apr 16, 2026

View reviewed changes

seanrathier and others added 3 commits April 16, 2026 14:29

Merge branch 'main' into feat/entity-maintainers-relationship-schema-…

f6d1517

…262893

seanrathier requested a review from CAWilson94 April 16, 2026 19:09

tiansivive approved these changes Apr 16, 2026

View reviewed changes

seanrathier enabled auto-merge (squash) April 16, 2026 19:16

Changes from node scripts/eslint_all_files --no-cache --fix

f5619b5

seanrathier merged commit 33ee38a into elastic:main Apr 16, 2026
14 checks passed

kibanamachine added the v9.5.0 label Apr 16, 2026

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Apr 16, 2026

[Entity Analytics] Update entity maintainers to use new relationship …

296cfcb

…schema (ids object) (elastic#262911) (cherry picked from commit 33ee38a)

kibanamachine mentioned this pull request Apr 16, 2026

[9.4] [Entity Analytics] Update entity maintainers to use new relationship schema (ids object) (#262911) #263917

Merged

Conversation

seanrathier commented Apr 13, 2026 • edited by kibanamachine Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

How to test

Uh oh!

elasticmachine commented Apr 13, 2026

Uh oh!

elasticmachine commented Apr 13, 2026

Uh oh!

elasticmachine commented Apr 13, 2026

Uh oh!

elasticmachine commented Apr 13, 2026

Uh oh!

macroscopeapp Bot commented Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Approvability

Uh oh!

tiansivive left a comment

Choose a reason for hiding this comment

Uh oh!

tiansivive Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

tiansivive Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

seanrathier Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

CAWilson94 Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

seanrathier Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

CAWilson94 left a comment

Choose a reason for hiding this comment

Uh oh!

tiansivive left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Apr 16, 2026

⏳ Build in-progress

History

Uh oh!

Uh oh!

kibanamachine commented Apr 16, 2026

Uh oh!

kibanamachine commented Apr 16, 2026

💚 All backports created successfully

Questions ?

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

seanrathier commented Apr 13, 2026 •

edited by kibanamachine

Loading

macroscopeapp Bot commented Apr 13, 2026 •

edited

Loading