[Fleet] Migrate inputs when migrate_from field is found by criamico · Pull Request #242934 · elastic/kibana

criamico · 2025-11-13T15:59:25Z

Related to elastic/package-spec#956
Package-spec PR: elastic/package-spec#1021

Summary

Migrate inputs and vars when migrate_from field is found.

Please note: for some integrations this migration path will require a change at elastic agent level. See elastic/package-spec#956 (comment)

Enable the new input and carries over its values from the old one
Inputs and variables that were enabled are now carried to the new version; however the state is maintained: if a variable is disabled, it stays disabled
Modified the secret handling so the existing secret references get carried over to the new inputs
Verified that this logic only run on package policy upgrade, when there is migrate_from present and Fleet detects a difference in input type
Add visual information about the migration

Example with a test integration:
https://github.com/user-attachments/assets/aaa56904-05d7-47b2-955b-5ae7a2c26c97

Testing

This was tested with a test version of sentinelOne (not finalized yet): https://github.com/elastic/integrations/compare/main...kcreddy:sentinel_one-migrate-inputs?expand=1

Screenshots

Checklist

Reviewers should verify this PR satisfies this list as well.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios

criamico · 2025-11-18T13:47:35Z

@elasticmachine merge upstream

criamico · 2025-11-24T13:47:12Z

@elasticmachine merge upstream

elasticmachine · 2026-02-25T10:22:36Z

Pinging @elastic/fleet (Team:Fleet)

criamico · 2026-02-26T14:31:06Z

@elasticmachine merge upstream

juliaElastic · 2026-02-27T12:03:38Z

x-pack/platform/plugins/shared/fleet/server/services/package_policy.ts

+      // input should never be auto-enabled or have user configuration carried into it.
+
+      let originalInputToMigrate: NewPackagePolicyInput | undefined;
+      if (update.migrate_from !== undefined && !update.deprecated) {


it would be nice to break down this logic to smaller functions

jillguyonnet · 2026-02-27T14:56:55Z

x-pack/platform/plugins/shared/fleet/server/services/package_policy.ts

+        update.streams = update.streams.map((newStream, idx) => {
+          let oldStream;
+
+          // Migrate stream-level vars by position since datasets differ between input types.


Do I understand this correctly in that we assume that the old and new input types have an equivalent set of streams in the same order? Or could it happen that the new input type has more or fewer streams than the old one? Should there be an explicit check for this?

Good question, I assumed that the migration is to an equivalent set of streams, but I have to ask the integration team. There was a similar discussion here but I'm not 100% sure about the streams.

jillguyonnet · 2026-02-27T15:07:19Z

x-pack/platform/plugins/shared/fleet/server/services/package_policy.ts

+      // input should never be auto-enabled or have user configuration carried into it.
+
+      let originalInputToMigrate: NewPackagePolicyInput | undefined;
+      if (update.migrate_from !== undefined && !update.deprecated) {


For my understanding: migrate_from doesn't seem to be cleaned up after migration, but that shouldn't be a concern on a future upgrade?

migrate_from comes directly from the package and it's not stored anywhere, so if the next version doesn't have it anymore it shouldn't be a concern.

jsoriano

Great, thanks!

...lic/applications/fleet/sections/agent_policy/edit_package_policy_page/components/upgrade.tsx

criamico · 2026-03-05T08:50:31Z

@elasticmachine merge upstream

criamico · 2026-03-11T10:29:09Z

@elasticmachine merge upstream

criamico · 2026-03-13T10:52:22Z

@elasticmachine merge upstream

criamico · 2026-03-18T13:39:10Z

@elasticmachine merge upstream

criamico · 2026-03-19T13:48:24Z

@elasticmachine merge upstream

criamico · 2026-03-19T21:26:09Z

@elasticmachine merge upstream

elasticmachine · 2026-03-19T21:58:59Z

⏳ Build in-progress

History

💔 Build #413048 failed 9adbfa2
💔 Build #412922 failed 5e2720d
💔 Build #412813 failed 3c886db
💔 Build #412312 failed 332ff23
💔 Build #412147 failed 191c66c
💚 Build #409616 succeeded 22dd1cd

cc @criamico

Related to elastic/package-spec#956 Package-spec PR: elastic/package-spec#1021 ## Summary Migrate inputs and vars when `migrate_from` field is found. **Please note:** for some integrations this migration path will require a change at elastic agent level. See elastic/package-spec#956 (comment) - Enable the new input and carries over its values from the old one - Inputs and variables that were enabled are now carried to the new version; however the state is maintained: if a variable is disabled, it stays disabled - Modified the secret handling so the existing secret references get carried over to the new inputs - Verified that this logic only run on package policy upgrade, when there is migrate_from present and Fleet detects a difference in input type - Add visual information about the migration Example with a test integration: https://github.com/user-attachments/assets/aaa56904-05d7-47b2-955b-5ae7a2c26c97 ### Testing This was tested with a test version of sentinelOne (not finalized yet): https://github.com/elastic/integrations/compare/main...kcreddy:sentinel_one-migrate-inputs?expand=1 ### Screenshots <img width="1641" height="1540" alt="Screenshot 2026-03-19 at 17 07 17" src="https://github.com/user-attachments/assets/819e46cd-823a-47df-8ef2-4fbce8900b53" /> <img width="1628" height="1524" alt="Screenshot 2026-03-19 at 17 07 43" src="https://github.com/user-attachments/assets/c5f97183-bcfd-496a-9882-e170323246e3" /> ### Checklist Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

[Fleet] Migrate inputs when migrate_from field is found

146c350

criamico self-assigned this Nov 13, 2025

criamico added the Team:Fleet Team label for Observability Data Collection Fleet team label Nov 13, 2025

elasticmachine and others added 2 commits November 18, 2025 14:47

Merge branch 'main' into poc_migrate_inputs

3926dac

Move logic to dryRun functionality

798b0ed

criamico mentioned this pull request Nov 20, 2025

[Change Proposal] Support migration to equivalent inputs elastic/package-spec#956

Closed

elasticmachine and others added 5 commits November 24, 2025 14:47

Merge branch 'main' into poc_migrate_inputs

e8f8f4c

Merge main and fix conflicts

4d24a02

Merge main and fix conflicts

cdefea8

Additional changes and unit tests

d9c8238

Auto-enable migrated vars

b46659c

criamico mentioned this pull request Feb 24, 2026

Add new field migrate_from to handle inputs migration elastic/package-spec#1021

Merged

2 tasks

Handle stream level migrations

aaf66a1

criamico marked this pull request as ready for review February 25, 2026 10:22

criamico requested a review from a team as a code owner February 25, 2026 10:22

Changes from make api-docs

f806b3c

elastic-vault-github-plugin-prod bot requested a review from a team as a code owner February 25, 2026 10:35

Merge main and solve conflicts

10ed50d

criamico added release_note:feature Makes this part of the condensed release notes backport:skip This PR does not require backporting v9.4.0 labels Feb 25, 2026

handle input deprecation case

32a7e8d

criamico requested a review from jsoriano February 25, 2026 11:11

criamico added the ci:cloud-deploy Create or update a Cloud deployment label Feb 26, 2026

Merge branch 'main' into poc_migrate_inputs

0f771ac

criamico added ci:cloud-deploy Create or update a Cloud deployment and removed ci:cloud-deploy Create or update a Cloud deployment labels Feb 27, 2026

juliaElastic reviewed Feb 27, 2026

View reviewed changes

jillguyonnet reviewed Feb 27, 2026

View reviewed changes

Break up package policy code to smaller functions

70e4a8f

mdbirnstiehl approved these changes Feb 27, 2026

View reviewed changes

jsoriano approved these changes Mar 3, 2026

View reviewed changes

...lic/applications/fleet/sections/agent_policy/edit_package_policy_page/components/upgrade.tsx Outdated Show resolved Hide resolved

Merge branch 'main' into poc_migrate_inputs

fc0ea0e

elasticmachine and others added 2 commits March 11, 2026 11:29

Merge branch 'main' into poc_migrate_inputs

5d164bd

Fixes based on end to end testing

f3e7b50

Merge branch 'main' into poc_migrate_inputs

22dd1cd

elasticmachine and others added 3 commits March 18, 2026 06:39

Merge branch 'main' into poc_migrate_inputs

191c66c

Add tooltip to package_policy_input_streams and update copy

332ff23

Fixes based on testing

3c886db

elasticmachine and others added 2 commits March 19, 2026 06:48

Merge branch 'main' into poc_migrate_inputs

5e2720d

Refactor common components

9adbfa2

Merge branch 'main' into poc_migrate_inputs

9679c5f

criamico merged commit fa0af58 into elastic:main Mar 20, 2026
18 checks passed

This was referenced Mar 31, 2026

migrate_from option from HTTPJSON to CEL input: input level variables are not migrated #260428

Open

migrate_from upgrade: migrated CEL input not auto-enabled, required vars not validated, agent enters degraded state #261398

Open

Conversation

criamico commented Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing

Screenshots

Checklist

Uh oh!

criamico commented Nov 18, 2025

Uh oh!

criamico commented Nov 24, 2025

Uh oh!

elasticmachine commented Feb 25, 2026

Uh oh!

criamico commented Feb 26, 2026

Uh oh!

juliaElastic Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

jillguyonnet Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

criamico Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

jillguyonnet Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

criamico Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

jsoriano left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

criamico commented Mar 5, 2026

Uh oh!

criamico commented Mar 11, 2026

Uh oh!

criamico commented Mar 13, 2026

Uh oh!

criamico commented Mar 18, 2026

Uh oh!

criamico commented Mar 19, 2026

Uh oh!

criamico commented Mar 19, 2026

Uh oh!

elasticmachine commented Mar 19, 2026

⏳ Build in-progress

History

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

criamico commented Nov 13, 2025 •

edited

Loading