[Fleet] generate OTel config for integration packages with otelcol inputs by juliaElastic · Pull Request #259968 · elastic/kibana

juliaElastic · 2026-03-27T11:59:55Z

Summary

Extends OTel collector configuration generation (generateOtelcolConfig) to support integration packages that include an otelcol input in their policy_templates.inputs[] array (coming from composable packages), as introduced in package-spec#1085.

Previously, OTel config generation worked only for input-type packages (packages with type: input and a single top-level input: otelcol). Integration packages with an otelcol input were not recognised by hasDynamicSignalTypes and did not receive dynamic signal-type routing.

Covered with unit and integration tests.

The test package contains this otelcol input in the manifest:

format_version: 3.6.0
name: integration_otel
title: Integration OTel Test Package
description: Test integration package with OpenTelemetry collector input
type: integration
version: 1.0.0
license: basic
categories:
  - observability
policy_templates:
  - name: integration_otel
    title: Integration OTel
    description: Collect data via OpenTelemetry collector
    inputs:
      - type: otelcol
        title: Collect OTel data
        description: Collect data via OpenTelemetry collector
        dynamic_signal_types: true
    data_streams:
      - otel_logs

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
Flaky Test Runner was used on any tests changed
The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
Review the backport guidelines and apply applicable backport:* labels.

Identify risks

Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging.

See some risk examples
...

elasticmachine · 2026-03-27T12:38:16Z

Pinging @elastic/fleet (Team:Fleet)

criamico · 2026-03-27T14:44:26Z

x-pack/platform/plugins/shared/fleet/server/services/epm/packages/input_type_packages.ts

+        (template as RegistryPolicyIntegrationTemplate).inputs?.some(
+          (input) => input.type === OTEL_COLLECTOR_INPUT_TYPE && input.dynamic_signal_types === true
+        )
+    ) === true


nit: I think you can remove the === true, since some returns a boolean

criamico · 2026-03-27T14:50:57Z

x-pack/platform/plugins/shared/fleet/server/services/epm/packages/input_type_packages.ts

+
+  // Check integration templates (type: integration packages) with otelcol in their inputs array
+  return (
+    packageInfo.policy_templates?.some(


Nit: I'm wondering if we should extract this condition to a function, something like IntegrationsWithOtelInputs for clarity and to be reused if needed elsewhere

If we create a separate function to list the templates that have otel inputs, we have to iterate again to determine which inputs have dynamic_signal_types. I would wait with this refactor until we have to reuse the list of templates somewhere else.

jsoriano · 2026-03-27T16:42:41Z

x-pack/platform/plugins/shared/fleet/common/types/models/epm.ts

@@ -341,6 +342,7 @@ export interface RegistryInput {
  [RegistryInputKeys.hide_in_var_group_options]?: Record<string, string[]>;
  [RegistryInputKeys.deprecated]?: DeprecationInfo;
  [RegistryInputKeys.migrate_from]?: string;
+  [RegistryInputKeys.dynamic_signal_types]?: boolean;


Are these the inputs in the main manifest? As the data stream type is not defined at this point how is this going to be interpreted in Fleet? Is it going to be applied to all streams using this input?

cc @teresaromero

Are these the inputs in the main manifest? As the data stream type is not defined at this point how is this going to be interpreted in Fleet?

Yes, it maps to the inputs array inside a policy_template in the main manifest.

Is it going to be applied to all streams using this input?

Yes it applies to all streams using that input. The flag is on the input definition, not on individual streams. Every stream in a policy that uses this input type will get the multi-signal-type treatment.

x-pack/platform/plugins/shared/fleet/server/services/epm/packages/input_type_packages.ts

macroscopeapp · 2026-03-30T07:32:24Z

Approvability

Verdict: Needs human review

This PR changes runtime behavior for OTel config generation and agent permission determination, moving from package-level to per-input dynamic signal type checking. All changed files are owned by @elastic/fleet (the author owns none of them), and there are open review comments questioning whether the implementation correctly handles edge cases with multiple inputs/policy templates.

^{You can customize Macroscope's approvability policy. Learn more.}

...plugins/shared/fleet/server/services/agent_policies/package_policies_to_agent_permissions.ts

x-pack/platform/plugins/shared/fleet/server/services/agent_policies/otel_collector.ts

… having dynamic signal types

criamico

LGTM 🚀

elasticmachine · 2026-03-30T10:12:52Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 9764845

Failed CI Steps

FTR Configs #93

Metrics [docs]

✅ unchanged

History

💔 Build #418552 failed b775cdd
💔 Build #417852 failed b960cc0

jsoriano

Thanks!

jsoriano · 2026-03-30T20:40:14Z

x-pack/platform/plugins/shared/fleet/server/services/agent_policies/otel_collector.test.ts

…e_for_children6 * commit '3402744f63ca1196e97b11ffac4e7f7efab240df': (80 commits) [PerUserAuth] Add EARS auth type for Connectors V2 (elastic#253695) Fix `@elastic/eui/require-aria-label-for-modals` lint violations across `@elastic/kibana-core` files (elastic#259757) [Entity Analytics][Leads generation][4] Add API routes, LeadDataClient, and async generation (elastic#257046) [Agent Builder] Agent-centric UX redesign (elastic#258005) fix query streams failing test (elastic#260277) [Lens as code] Add list layout to the new API (elastic#259967) [FTR] Add warning comments to deployment-agnostic FTR base configs (elastic#260018) [Discover][Logs profile] Fix missing search highlights (elastic#260056) Plugin system: safe deletion (elastic#259038) [Infra] Fix Hosts filter options to match selected schema (elastic#259825) Manual Entity Resolution and flyout representation (elastic#260162) [Cascade] Handle grouping on fields with unset values (elastic#260033) [Fleet] generate OTel config for integration packages with otelcol inputs (elastic#259968) [Search] Switch over to V2 index management details (elastic#259866) [inference] increase timeout for ES inference calls (elastic#260382) [ES|QL] Enable subqueries (elastic#257455) [ES|QL] Change Point order free options (elastic#260282) [Auth] Added authentication strategy for UIAM OAuth (elastic#256182) [Security Solution] Add "alerts_candidate_count" rule execution metric (elastic#259917) [api-docs] 2026-03-31 Daily api_docs build (elastic#260380) ...

…puts (elastic#259968) ## Summary Closes elastic#252949 Extends OTel collector configuration generation (`generateOtelcolConfig`) to support integration packages that include an `otelcol` input in their `policy_templates.inputs[]` array (coming from composable packages), as introduced in [package-spec#1085](elastic/package-spec#1085). Previously, OTel config generation worked only for input-type packages (packages with `type: input` and a single top-level `input: otelcol`). Integration packages with an otelcol input were not recognised by `hasDynamicSignalTypes` and did not receive dynamic signal-type routing. Covered with unit and integration tests. The test package contains this otelcol input in the manifest: ``` format_version: 3.6.0 name: integration_otel title: Integration OTel Test Package description: Test integration package with OpenTelemetry collector input type: integration version: 1.0.0 license: basic categories: - observability policy_templates: - name: integration_otel title: Integration OTel description: Collect data via OpenTelemetry collector inputs: - type: otelcol title: Collect OTel data description: Collect data via OpenTelemetry collector dynamic_signal_types: true data_streams: - otel_logs ``` ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [ ] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ...

generate Otel config for integration packages with otelcol inputs

714b353

juliaElastic added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting labels Mar 27, 2026

juliaElastic changed the title ~~[Fleet] generate Otel config for integration packages with otelcol inputs~~ [Fleet] generate OTel config for integration packages with otelcol inputs Mar 27, 2026

juliaElastic marked this pull request as ready for review March 27, 2026 12:17

juliaElastic requested a review from a team as a code owner March 27, 2026 12:17

compacting tests

b960cc0

botelastic bot added the Team:Fleet Team label for Observability Data Collection Fleet team label Mar 27, 2026

juliaElastic force-pushed the otel-config-integration-package branch from 8aef54d to b960cc0 Compare March 27, 2026 14:33

juliaElastic added 2 commits March 27, 2026 15:45

handle 429 error

48ebe11

refactor

8394336

criamico reviewed Mar 27, 2026

View reviewed changes

remove === true

07ece80

jsoriano reviewed Mar 27, 2026

View reviewed changes

Merge branch 'main' into otel-config-integration-package

b775cdd

juliaElastic added 2 commits March 30, 2026 09:49

fix conflict resolution

619ad47

refactored to check dynamic signal type for specific templates

96f841e

macroscopeapp bot reviewed Mar 30, 2026

View reviewed changes

...plugins/shared/fleet/server/services/agent_policies/package_policies_to_agent_permissions.ts Outdated Show resolved Hide resolved

x-pack/platform/plugins/shared/fleet/server/services/agent_policies/otel_collector.ts Outdated Show resolved Hide resolved

juliaElastic added 2 commits March 30, 2026 10:30

add tests to cover one template with two inputs with only one of them…

5b523e3

… having dynamic signal types

address review comments

9764845

juliaElastic requested review from criamico and jsoriano March 30, 2026 08:56

criamico approved these changes Mar 30, 2026

View reviewed changes

jsoriano approved these changes Mar 30, 2026

View reviewed changes

x-pack/platform/plugins/shared/fleet/server/services/agent_policies/otel_collector.test.ts

Copy link
Copy Markdown

Member

jsoriano Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

juliaElastic merged commit f84b69f into elastic:main Mar 31, 2026
20 checks passed

kibanamachine added the v9.4.0 label Mar 31, 2026

teresaromero mentioned this pull request Apr 6, 2026

[updatecli] Update latest snapshot to 9.4.0-88e3ed81 elastic/elastic-package#3430

Closed

Conversation

juliaElastic commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Checklist

Identify risks

Uh oh!

elasticmachine commented Mar 27, 2026

Uh oh!

criamico Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

criamico Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

juliaElastic Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jsoriano Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

juliaElastic Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

juliaElastic Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

macroscopeapp bot commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Approvability

Uh oh!

Uh oh!

Uh oh!

criamico left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Mar 30, 2026

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

History

Uh oh!

jsoriano left a comment

Choose a reason for hiding this comment

Uh oh!

jsoriano Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

juliaElastic commented Mar 27, 2026 •

edited

Loading

juliaElastic Mar 27, 2026 •

edited

Loading

macroscopeapp bot commented Mar 30, 2026 •

edited

Loading