Skip to content

[UII] Use explicit fleet server & output IDs for agentless on ECH and Serverless#225188

Merged
jen-huang merged 20 commits intoelastic:mainfrom
jen-huang:feat/agentless-hosts
Oct 15, 2025
Merged

[UII] Use explicit fleet server & output IDs for agentless on ECH and Serverless#225188
jen-huang merged 20 commits intoelastic:mainfrom
jen-huang:feat/agentless-hosts

Conversation

@jen-huang
Copy link
Contributor

@jen-huang jen-huang commented Jun 24, 2025

Summary

Resolves https://github.com/elastic/ingest-dev/issues/5924.

This PR adjusts the fixed IDs for Fleet Server host and ES output used by agentless policies on ECH. This is done by programmatically creating new uneditable host objects during Fleet setup.

On serverless, the logic remains the same: serverless agent policies use preconfigured hosts IDs defined by project controller.

PR also:

  • Fixes minor typos and adds conditionals for clarity
  • Applies the same fixed ID to agentless policies' monitoring output ID (previously only integration data output ID was set)

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

  • Unit or functional tests were updated or added to match the most common scenarios
  • The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

@jen-huang jen-huang self-assigned this Jun 24, 2025
@jen-huang jen-huang added release_note:skip Skip the PR/issue when compiling release notes Team:Fleet Team label for Observability Data Collection Fleet team labels Jun 24, 2025
@jen-huang jen-huang changed the title [WIP][UII] Define explicit fleet server & output IDs for agentless on ECH and Serverless [UII] Use explicit fleet server & output IDs for agentless on ECH and Serverless Oct 9, 2025
@jen-huang jen-huang added ci:cloud-deploy Create or update a Cloud deployment ci:project-deploy-security Create a Security Serverless Project backport:version Backport to applied version labels v9.3.0 labels Oct 9, 2025
@jen-huang jen-huang marked this pull request as ready for review October 9, 2025 19:21
@jen-huang jen-huang requested a review from a team as a code owner October 9, 2025 19:21
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@jen-huang jen-huang requested a review from MichelLosier October 9, 2025 19:59
@nchaulet nchaulet self-requested a review October 10, 2025 13:05
@nchaulet
Copy link
Member

I am not sure this will totally work as we delete/ non used preconfigured output/fleet server hosts during Fleet setup, we continuously will try to delete those, or mark them as non preconfigured if they are used by a policy

You can see in log

[2025-10-10T09:10:44.406-04:00][INFO ][plugins.fleet] Deleting preconfigured output es-agentless-output

I think one way to prevent that will be to add the newly created fleet server host to the preconfigured list here and same for output here

Copy link
Contributor

@MichelLosier MichelLosier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good so far, good catch @nchaulet on the clean up part

@jen-huang
Copy link
Contributor Author

I think one way to prevent that will be to add the newly created fleet server host to the preconfigured list here and same for output here

@nchaulet Thanks for catching! That explains some of the test failures. I opted to skip these IDs in the cleanup functions themselves in 9f263a1, WDYT?

@kibanamachine kibanamachine added backport:skip This PR does not require backporting and removed backport:version Backport to applied version labels labels Oct 15, 2025
ersin-erdal pushed a commit to ersin-erdal/kibana that referenced this pull request Oct 15, 2025
@jen-huang jen-huang deleted the feat/agentless-hosts branch October 15, 2025 17:32
@jen-huang jen-huang added v9.0.0 backport:version Backport to applied version labels v9.2.0 v9.1.6 v8.19.6 and removed backport:skip This PR does not require backporting labels Oct 15, 2025
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.19, 9.0, 9.1, 9.2

https://github.com/elastic/kibana/actions/runs/18537454866

@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.19, 9.0, 9.1, 9.2

https://github.com/elastic/kibana/actions/runs/18537455061

@kibanamachine
Copy link
Contributor

💔 Some backports could not be created

Status Branch Result
8.19 Backport failed because of merge conflicts

You might need to backport the following PRs to 8.19:
- [Cloud Security] Add Cloud Connectors CSPM Support (#212200)
- [Cloud Connector] Add cloud_connectors config in Agentless API (#215421)
9.0 Backport failed because of merge conflicts
9.1
9.2

Note: Successful backport PRs will be merged automatically after passing CI.

Manual backport

To create the backport manually run:

node scripts/backport --pr 225188

Questions ?

Please refer to the Backport tool documentation

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 15, 2025
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 15, 2025
@kibanamachine
Copy link
Contributor

💔 Some backports could not be created

Status Branch Result
8.19 Backport failed because of merge conflicts

You might need to backport the following PRs to 8.19:
- [Cloud Security] Add Cloud Connectors CSPM Support (#212200)
- [Cloud Connector] Add cloud_connectors config in Agentless API (#215421)
9.0 Backport failed because of merge conflicts
9.1
9.2

Note: Successful backport PRs will be merged automatically after passing CI.

Manual backport

To create the backport manually run:

node scripts/backport --pr 225188

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Oct 15, 2025
…CH and Serverless (#225188) (#239212)

# Backport

This will backport the following commits from `main` to `9.2`:
- [[UII] Use explicit fleet server & output IDs for agentless on ECH and
Serverless (#225188)](#225188)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Jen
Huang","email":"its.jenetic@gmail.com"},"sourceCommit":{"committedDate":"2025-10-15T02:59:59Z","message":"[UII]
Use explicit fleet server & output IDs for agentless on ECH and
Serverless
(#225188)","sha":"237a489648c48ab43701b77a0af380f5a7360cee","branchLabelMapping":{"^v9.3.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Fleet","v9.0.0","ci:cloud-deploy","ci:project-deploy-security","backport:version","v9.2.0","v9.3.0","v9.1.6","v8.19.6"],"title":"[UII]
Use explicit fleet server & output IDs for agentless on ECH and
Serverless","number":225188,"url":"https://github.com/elastic/kibana/pull/225188","mergeCommit":{"message":"[UII]
Use explicit fleet server & output IDs for agentless on ECH and
Serverless
(#225188)","sha":"237a489648c48ab43701b77a0af380f5a7360cee"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","9.2","9.1","8.19"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.2","label":"v9.2.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.3.0","branchLabelMappingKey":"^v9.3.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/225188","number":225188,"mergeCommit":{"message":"[UII]
Use explicit fleet server & output IDs for agentless on ECH and
Serverless
(#225188)","sha":"237a489648c48ab43701b77a0af380f5a7360cee"}},{"branch":"9.1","label":"v9.1.6","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.19","label":"v8.19.6","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Jen Huang <its.jenetic@gmail.com>
jen-huang added a commit that referenced this pull request Oct 15, 2025
…CH and Serverless (#225188) (#239231)

# Backport

This will backport the following commits from `main` to `9.1`:
- [[UII] Use explicit fleet server & output IDs for agentless on ECH and
Serverless (#225188)](#225188)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Jen
Huang","email":"its.jenetic@gmail.com"},"sourceCommit":{"committedDate":"2025-10-15T02:59:59Z","message":"[UII]
Use explicit fleet server & output IDs for agentless on ECH and
Serverless
(#225188)","sha":"237a489648c48ab43701b77a0af380f5a7360cee","branchLabelMapping":{"^v9.3.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Fleet","v9.0.0","ci:cloud-deploy","ci:project-deploy-security","backport:version","v9.2.0","v9.3.0","v9.1.6","v8.19.6"],"title":"[UII]
Use explicit fleet server & output IDs for agentless on ECH and
Serverless","number":225188,"url":"https://github.com/elastic/kibana/pull/225188","mergeCommit":{"message":"[UII]
Use explicit fleet server & output IDs for agentless on ECH and
Serverless
(#225188)","sha":"237a489648c48ab43701b77a0af380f5a7360cee"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","9.2","9.1","8.19"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.2","label":"v9.2.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.3.0","branchLabelMappingKey":"^v9.3.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/225188","number":225188,"mergeCommit":{"message":"[UII]
Use explicit fleet server & output IDs for agentless on ECH and
Serverless
(#225188)","sha":"237a489648c48ab43701b77a0af380f5a7360cee"}},{"branch":"9.1","label":"v9.1.6","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.19","label":"v8.19.6","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
mgadewoll pushed a commit to tkajtoch/kibana that referenced this pull request Oct 17, 2025
rylnd pushed a commit to rylnd/kibana that referenced this pull request Oct 17, 2025
nickpeihl pushed a commit to nickpeihl/kibana that referenced this pull request Oct 23, 2025
NicholasPeretti pushed a commit to NicholasPeretti/kibana that referenced this pull request Oct 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:version Backport to applied version labels ci:cloud-deploy Create or update a Cloud deployment ci:project-deploy-security Create a Security Serverless Project release_note:skip Skip the PR/issue when compiling release notes Team:Fleet Team label for Observability Data Collection Fleet team v9.1.6 v9.2.0 v9.3.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants