Support custom data_stream.type in system and policy tests#3509
Merged
jsoriano merged 4 commits intoelastic:mainfrom Apr 29, 2026
Merged
Support custom data_stream.type in system and policy tests#3509jsoriano merged 4 commits intoelastic:mainfrom
jsoriano merged 4 commits intoelastic:mainfrom
Conversation
5 tasks
jsoriano
commented
Apr 29, 2026
| Requires Kibana 8.18.3 / 8.19.0 / 9.1.0 or later (Fleet PR [#214216](https://github.com/elastic/kibana/pull/214216)). | ||
| Because the Kibana simplified Fleet API does not yet accept `data_stream.type` as a | ||
| stream-level variable (it rejects unknown vars), tests that use this override must also | ||
| set `policy_api_format: legacy` until a Kibana fix is available. |
Member
Author
There was a problem hiding this comment.
Follow up issue created for that: elastic/kibana#266321
mrodm
approved these changes
Apr 29, 2026
Collaborator
💚 Build Succeeded
History
cc @jsoriano |
mrodm
approved these changes
Apr 29, 2026
teresaromero
approved these changes
Apr 29, 2026
Contributor
teresaromero
left a comment
teresaromero
left a comment
There was a problem hiding this comment.
why we would want to change the type a datastream gets?
Member
Author
With input packages a user may want to chose the signal type they are collecting. Imagine for example querying HTTP endpoints with httpjson. This input package is defined with the "logs" type, but you could also collect metrics using HTTP JSON endpoints. With this feature we allow users to store the data they are collecting with the type they consider to be better suited. Here we are allowing package developers to test their packages with this feature. |
teresaromero
added a commit
that referenced
this pull request
May 7, 2026
…3480) Composable packages with requires.input can have multiple required inputs of the same type. package-spec#1135 (3.6.0+) requires distinct policy_templates[].inputs[].name and streams[].input that reference that name—not a single derived id for both policy and streams. elastic-package previously did not match that pattern, which caused ambiguous Fleet policies and errors (e.g. missing stream templates). This branch also builds policies from the built package manifest (so composable package: refs resolve for Fleet), adds CI/fixtures (nginx composable, dual same-type inputs, bootstrap input pkgs), and includes data_stream.type for input-package system/policy tests (#3509) plus docs for Kibana/Fleet version behavior.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
elastic/package-spec#849 introduced a way to override the data stream type for input packages. Users can use this toggle to decide the data stream to use for data collected with generic input packages.
Allow to override the data stream type using the
data_stream.typevariable in system and policy tests.