[Automatic Migration] Introduce Last execution info for a migration

[logeekal]

Resolves

• https://github.com/elastic/security-team/issues/12681
• https://github.com/elastic/security-team/issues/12682

Summary

Please find the summary of all the changes below.

Last Execution Info

This Enhancement introduces last_execution object to the migration document. There is some information that we need to add for the last run migration ( eg. error occured or connector_id used ). This PR add that info. Below is how a sample migration document looks like with last_execution info and shape of last_execution object.

export const RuleMigrationLastExecution = z.object({
  /**
   * The moment the last execution started.
   */
  started_at: z.string().optional(),
  /**
   * The moment the last execution ended.
   */
  ended_at: z.string().optional(),
  /**
   * The connector ID used for the last execution, if applicable.
   */
  connector_id: z.string().optional(),
  /**
   * The error message if the last execution failed.
   */
  error: z.string().optional(),
  /**
   * Indicates if the last execution was aborted by the user.
   */
  is_aborted: z.boolean().optional(),
});

{
  "created_by": "u_v6jVi3qZ21kAEUwmbl-sfujRnwydrltycxyByl3jtIM_0",
  "created_at": "2025-05-28T06:10:37.638Z",
  "last_execution": {
    "connector_id": "azureOpenAiGPT4o",
    "started_at": "2025-05-28T06:10:48.030Z",
    "is_aborted": false,
    "error": "Failed to populate ELSER indices. Make sure the ELSER model is deployed and running at Machine Learning > Trained Models. Error: Exception when running inference id [.elser-2-elasticsearch] on field [elser_embedding]",
    "ended_at": "2025-05-28T06:11:31.464Z"
}

Why Last Execution info was needed?

• An automatic migration can run into an error and terminate. Today, we store that error in server state and do not persist. This can be a problem when server is restarted and user will not be able to see that error.

  • So, we need a way to persist the error for last execution.

• Additionally, we do not have any details on when the last execution was started and finished.

• We also need to store the connector_id so that we can re-use it when user is restarting a migration.

In addition to resolving above issues, this Ticket makes some minor UI Changes to the Migration Ready Panel in different states of a Migration

When Migration is Ready to be started.

This remains exactly the same as before:

When Migration was Aborted

Before	After

When there was an error

Before	After

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
• This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
• Flaky Test Runner was used on any tests changed
• The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

[logeekal]

Even if pending === total but the task was aborted, we would want to inform user that the task was aborted but can still be started.

[elasticmachine]

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

[logeekal]

@semd , as we discussed, here are some cosmetic naming changes + final workflow for Saving aborted/failed/ended state in migration index

6930b3b

[semd]

LGTM! Thanks Jatin

[kibanamachine]

Starting backport for target branches: 8.19

https://github.com/elastic/kibana/actions/runs/15410758137

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 8655814

Failed CI Steps

• FTR Configs #6
• FTR Configs #55
• FTR Configs #73
• FTR Configs #81
• FTR Configs #127

Test Failures

• [job] [logs] FTR Configs #73 / Rule execution logic API - Trial License/Complete Tier @ess @serverless synthetic source synthetic source limitations should convert dot-notation to nested objects
• [job] [logs] FTR Configs #55 / Security Solution - Telemetry Security Telemetry - Indices metadata task telemetry @ess indices metadata should include ilm_policy in data stream events when defined

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
securitySolution	9.2MB	9.2MB	+1.3KB

History

• 💔 Build #304135 failed 6bf6431
• 💛 Build #304068 was flaky 2402a9c
• 💔 Build #303716 failed f27f6ba
• 💔 Build #303672 failed de7721c
• 💔 Build #303662 failed 5b15835

[kibanamachine]

💔 All backports failed

Status	Branch	Result
❌	8.19	Backport failed because of merge conflicts You might need to backport the following PRs to 8.19: - [Security Solution] Integrations card in automatic migrations (#221493)

Manual backport

To create the backport manually run:

node scripts/backport --pr 221635

Questions ?

Please refer to the Backport tool documentation

[kibanamachine]

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 221635 locally
cc: @logeekal

[kibanamachine]

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 221635 locally
cc: @logeekal

[kibanamachine]

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 221635 locally
cc: @logeekal

[kibanamachine]

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 221635 locally
cc: @logeekal

[kibanamachine]

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 221635 locally
cc: @logeekal

[kibanamachine]

Starting backport for target branches: 8.19

https://github.com/elastic/kibana/actions/runs/15581696693

[kibanamachine]

💚 All backports created successfully

Status	Branch	Result
✅	8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation