Skip to content

[Security Solution][SIEM Migrations] Add name field to rule migrations ui and data model#223860

Merged
semd merged 36 commits intoelastic:mainfrom
kqualters-elastic:siem-migrations-name
Jun 20, 2025
Merged

[Security Solution][SIEM Migrations] Add name field to rule migrations ui and data model#223860
semd merged 36 commits intoelastic:mainfrom
kqualters-elastic:siem-migrations-name

Conversation

@kqualters-elastic
Copy link
Contributor

@kqualters-elastic kqualters-elastic commented Jun 13, 2025 •
edited by kibanamachine
Loading

Summary

This pr adds the ability to set and edit the name field for automatic rule migrations, done in either the initial data upload workflow, or at any step thereafter in the migration list cards. Also adds logic to migrate existing migration metadata docs if they do not have a name, intended to match what the user saw previously, the string 'SIEM Migration #{$indexNumber}' users saw before this change.

Kibana logs when migrating a migration lacking a name:
image
Edit an existing migration name:
image
image
New sub step in rules upload form:
image
Migrated name:
image
(shown in dev tools because the migration does not have any rules associated, and so the card list does not include it)

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

  • Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
  • Unit or functional tests were updated or added to match the most common scenarios
  • This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.

@kqualters-elastic kqualters-elastic added release_note:feature Makes this part of the condensed release notes Team:Threat Hunting:Investigations Security Solution Threat Hunting Investigations Team backport:version Backport to applied version labels v9.1.0 v8.19.0 labels Jun 13, 2025
@kqualters-elastic kqualters-elastic marked this pull request as ready for review June 13, 2025 22:30
@kqualters-elastic kqualters-elastic requested a review from a team as a code owner June 13, 2025 22:30
@elasticmachine
Copy link
Contributor

elasticmachine commented Jun 13, 2025

Pinging @elastic/security-threat-hunting-investigations (Team:Threat Hunting:Investigations)

@kqualters-elastic kqualters-elastic changed the title WIP [Security Solution][SIEM Migrations] Add name field to rule migrations ui and data model [Security Solution][SIEM Migrations] Add name field to rule migrations ui and data model Jun 13, 2025
logeekal
logeekal reviewed Jun 17, 2025
View reviewed changes
...rity_solution/server/lib/siem_migrations/rules/data/rule_migrations_data_migration_client.ts Outdated
params,
}: {
id: string;
params: Partial<StoredSiemMigration>;
Copy link
Contributor

@logeekal logeekal Jun 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
params: Partial<StoredSiemMigration>;
params: UpdateRuleMigrationRequestBody

logeekal
logeekal reviewed Jun 17, 2025
View reviewed changes
...server/lib/siem_migrations/rules/index_migrators/rule_migrations_per_space_index_migrator.ts Outdated
);
}

const migrationsMaybeMissingName = existingMigrationsFromMigrationsIndex
Copy link
Contributor

@logeekal logeekal Jun 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think name migrations should be separate from migrateRuleMigrationIndex and it should run after that.

This is because migrateRuleMigrationIndex creates migration index and its missing documents. Once that is done then migrationRuleMigrationName( or something like that) operation should run for all docs that do not have name.

Copy link
Contributor

@logeekal logeekal Jun 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we do it this way, name will not be populated where this migration has already run, because they will not be filtered based on the logic of line 139.

Copy link
Contributor Author

@kqualters-elastic kqualters-elastic Jun 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is outside of the if statement on 142, so it should still run I think.

logeekal
logeekal approved these changes Jun 20, 2025
View reviewed changes
Copy link
Contributor

@logeekal logeekal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you both @semd and @kqualters-elastic

@semd semd enabled auto-merge (squash) June 20, 2025 15:23
@semd semd merged commit 9af4a74 into elastic:main Jun 20, 2025
10 checks passed
@kibanamachine
Copy link
Contributor

kibanamachine commented Jun 20, 2025

Starting backport for target branches: 8.19

https://github.com/elastic/kibana/actions/runs/15785754798

@elasticmachine
Copy link
Contributor

elasticmachine commented Jun 20, 2025

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
securitySolution 7649 7655 +6

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
securitySolution 9.4MB 9.4MB +4.4KB

History

kibanamachine added a commit to kibanamachine/kibana that referenced this pull request Jun 20, 2025
@kqualters-elastic @kibanamachine @semd
[Security Solution][SIEM Migrations] Add name field to rule migration…
ea6d7f1 
…s ui and data model (elastic#223860)

## Summary
This pr adds the ability to set and edit the name field for automatic
rule migrations, done in either the initial data upload workflow, or at
any step thereafter in the migration list cards. Also adds logic to
migrate existing migration metadata docs if they do not have a name,
intended to match what the user saw previously, the string 'SIEM
Migration \#{$indexNumber}' users saw before this change.

Kibana logs when migrating a migration lacking a name:

![image](https://github.com/user-attachments/assets/53e0a2ac-eea9-43cb-ac2e-5bb1858ac257)
Edit an existing migration name:

![image](https://github.com/user-attachments/assets/2c71b773-cadb-48a3-8363-24ce1988070e)

![image](https://github.com/user-attachments/assets/2d83c23c-4d9d-4eef-ba86-252abf6f1dea)
New sub step in rules upload form:

![image](https://github.com/user-attachments/assets/2fc59cf4-e72b-463a-9968-8c36345e5a16)
Migrated name:

![image](https://github.com/user-attachments/assets/b8543957-0c1b-44bc-a68f-2b5663263010)
(shown in dev tools because the migration does not have any rules
associated, and so the card list does not include it)

### Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Sergi Massaneda <sergi.massaneda@elastic.co>
(cherry picked from commit 9af4a74)
@kibanamachine kibanamachine mentioned this pull request Jun 20, 2025
Merged
@kibanamachine
Copy link
Contributor

kibanamachine commented Jun 20, 2025

💚 All backports created successfully

Status Branch Result
✅ 8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

@kibanamachine kibanamachine mentioned this pull request Jun 20, 2025
Merged
kibanamachine added a commit that referenced this pull request Jun 20, 2025
@kibanamachine @kqualters-elastic @semd
[8.19] [Security Solution][SIEM Migrations] Add name field to rule mi…
b405d3a 
…grations ui and data model (#223860) (#224754)

# Backport

This will backport the following commits from `main` to `8.19`:
- [[Security Solution][SIEM Migrations] Add name field to rule
migrations ui and data model
(#223860)](#223860)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Kevin
Qualters","email":"56408403+kqualters-elastic@users.noreply.github.com"},"sourceCommit":{"committedDate":"2025-06-20T18:51:24Z","message":"[Security
Solution][SIEM Migrations] Add name field to rule migrations ui and data
model (#223860)\n\n## Summary\nThis pr adds the ability to set and edit
the name field for automatic\nrule migrations, done in either the
initial data upload workflow, or at\nany step thereafter in the
migration list cards. Also adds logic to\nmigrate existing migration
metadata docs if they do not have a name,\nintended to match what the
user saw previously, the string 'SIEM\nMigration \\#{$indexNumber}'
users saw before this change.\n\nKibana logs when migrating a migration
lacking a
name:\n\n![image](https://github.com/user-attachments/assets/53e0a2ac-eea9-43cb-ac2e-5bb1858ac257)\nEdit
an existing migration
name:\n\n![image](https://github.com/user-attachments/assets/2c71b773-cadb-48a3-8363-24ce1988070e)\n\n![image](https://github.com/user-attachments/assets/2d83c23c-4d9d-4eef-ba86-252abf6f1dea)\nNew
sub step in rules upload
form:\n\n![image](https://github.com/user-attachments/assets/2fc59cf4-e72b-463a-9968-8c36345e5a16)\nMigrated
name:\n\n![image](https://github.com/user-attachments/assets/b8543957-0c1b-44bc-a68f-2b5663263010)\n(shown
in dev tools because the migration does not have any rules\nassociated,
and so the card list does not include it)\n\n\n### Checklist\n\nCheck
the PR satisfies following conditions. \n\nReviewers should verify this
PR satisfies this list as well.\n\n- [x] Any text added follows [EUI's
writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\nsentence case text and includes
[i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n-
[x] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common scenarios\n- [ ] This was
checked for breaking HTTP API changes, and any breaking\nchanges have
been approved by the breaking-change committee.
The\n`release_note:breaking` label should be applied in these
situations.\n\n---------\n\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by: Sergi
Massaneda
<sergi.massaneda@elastic.co>","sha":"9af4a749d0fa8bbb8fd9136e02627c8beb03f076","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:feature","Team:Threat
Hunting:Investigations","backport:version","v9.1.0","v8.19.0"],"title":"[Security
Solution][SIEM Migrations] Add name field to rule migrations ui and data
model","number":223860,"url":"https://github.com/elastic/kibana/pull/223860","mergeCommit":{"message":"[Security
Solution][SIEM Migrations] Add name field to rule migrations ui and data
model (#223860)\n\n## Summary\nThis pr adds the ability to set and edit
the name field for automatic\nrule migrations, done in either the
initial data upload workflow, or at\nany step thereafter in the
migration list cards. Also adds logic to\nmigrate existing migration
metadata docs if they do not have a name,\nintended to match what the
user saw previously, the string 'SIEM\nMigration \\#{$indexNumber}'
users saw before this change.\n\nKibana logs when migrating a migration
lacking a
name:\n\n![image](https://github.com/user-attachments/assets/53e0a2ac-eea9-43cb-ac2e-5bb1858ac257)\nEdit
an existing migration
name:\n\n![image](https://github.com/user-attachments/assets/2c71b773-cadb-48a3-8363-24ce1988070e)\n\n![image](https://github.com/user-attachments/assets/2d83c23c-4d9d-4eef-ba86-252abf6f1dea)\nNew
sub step in rules upload
form:\n\n![image](https://github.com/user-attachments/assets/2fc59cf4-e72b-463a-9968-8c36345e5a16)\nMigrated
name:\n\n![image](https://github.com/user-attachments/assets/b8543957-0c1b-44bc-a68f-2b5663263010)\n(shown
in dev tools because the migration does not have any rules\nassociated,
and so the card list does not include it)\n\n\n### Checklist\n\nCheck
the PR satisfies following conditions. \n\nReviewers should verify this
PR satisfies this list as well.\n\n- [x] Any text added follows [EUI's
writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\nsentence case text and includes
[i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n-
[x] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common scenarios\n- [ ] This was
checked for breaking HTTP API changes, and any breaking\nchanges have
been approved by the breaking-change committee.
The\n`release_note:breaking` label should be applied in these
situations.\n\n---------\n\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by: Sergi
Massaneda
<sergi.massaneda@elastic.co>","sha":"9af4a749d0fa8bbb8fd9136e02627c8beb03f076"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/223860","number":223860,"mergeCommit":{"message":"[Security
Solution][SIEM Migrations] Add name field to rule migrations ui and data
model (#223860)\n\n## Summary\nThis pr adds the ability to set and edit
the name field for automatic\nrule migrations, done in either the
initial data upload workflow, or at\nany step thereafter in the
migration list cards. Also adds logic to\nmigrate existing migration
metadata docs if they do not have a name,\nintended to match what the
user saw previously, the string 'SIEM\nMigration \\#{$indexNumber}'
users saw before this change.\n\nKibana logs when migrating a migration
lacking a
name:\n\n![image](https://github.com/user-attachments/assets/53e0a2ac-eea9-43cb-ac2e-5bb1858ac257)\nEdit
an existing migration
name:\n\n![image](https://github.com/user-attachments/assets/2c71b773-cadb-48a3-8363-24ce1988070e)\n\n![image](https://github.com/user-attachments/assets/2d83c23c-4d9d-4eef-ba86-252abf6f1dea)\nNew
sub step in rules upload
form:\n\n![image](https://github.com/user-attachments/assets/2fc59cf4-e72b-463a-9968-8c36345e5a16)\nMigrated
name:\n\n![image](https://github.com/user-attachments/assets/b8543957-0c1b-44bc-a68f-2b5663263010)\n(shown
in dev tools because the migration does not have any rules\nassociated,
and so the card list does not include it)\n\n\n### Checklist\n\nCheck
the PR satisfies following conditions. \n\nReviewers should verify this
PR satisfies this list as well.\n\n- [x] Any text added follows [EUI's
writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\nsentence case text and includes
[i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n-
[x] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common scenarios\n- [ ] This was
checked for breaking HTTP API changes, and any breaking\nchanges have
been approved by the breaking-change committee.
The\n`release_note:breaking` label should be applied in these
situations.\n\n---------\n\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by: Sergi
Massaneda
<sergi.massaneda@elastic.co>","sha":"9af4a749d0fa8bbb8fd9136e02627c8beb03f076"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Kevin Qualters <56408403+kqualters-elastic@users.noreply.github.com>
Co-authored-by: Sergi Massaneda <sergi.massaneda@elastic.co>
akowalska622 pushed a commit to akowalska622/kibana that referenced this pull request Jun 25, 2025
@kqualters-elastic @kibanamachine
@semd @akowalska622
[Security Solution][SIEM Migrations] Add name field to rule migration…
7db2738 
…s ui and data model (elastic#223860)

## Summary
This pr adds the ability to set and edit the name field for automatic
rule migrations, done in either the initial data upload workflow, or at
any step thereafter in the migration list cards. Also adds logic to
migrate existing migration metadata docs if they do not have a name,
intended to match what the user saw previously, the string 'SIEM
Migration \#{$indexNumber}' users saw before this change.

Kibana logs when migrating a migration lacking a name:

![image](https://github.com/user-attachments/assets/53e0a2ac-eea9-43cb-ac2e-5bb1858ac257)
Edit an existing migration name:

![image](https://github.com/user-attachments/assets/2c71b773-cadb-48a3-8363-24ce1988070e)

![image](https://github.com/user-attachments/assets/2d83c23c-4d9d-4eef-ba86-252abf6f1dea)
New sub step in rules upload form:

![image](https://github.com/user-attachments/assets/2fc59cf4-e72b-463a-9968-8c36345e5a16)
Migrated name:

![image](https://github.com/user-attachments/assets/b8543957-0c1b-44bc-a68f-2b5663263010)
(shown in dev tools because the migration does not have any rules
associated, and so the card list does not include it)


### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Sergi Massaneda <sergi.massaneda@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@logeekal logeekal logeekal approved these changes

@semd semd semd approved these changes

Assignees

No one assigned

Labels

backport:version Backport to applied version labels release_note:feature Makes this part of the condensed release notes Team:Threat Hunting:Investigations Security Solution Threat Hunting Investigations Team v8.19.0 v9.1.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@kqualters-elastic @elasticmachine @semd @kibanamachine @logeekal