Skip to content

feat: replace input package references in stream definitions#3

Draft
teresaromero wants to merge 2 commits into3280-bundle-fieldsfrom
3380-replace-stream-packages
Draft

feat: replace input package references in stream definitions#3
teresaromero wants to merge 2 commits into3280-bundle-fieldsfrom
3380-replace-stream-packages

Conversation

@teresaromero
Copy link
Copy Markdown
Owner

@teresaromero teresaromero commented Mar 24, 2026

Replace input package references with resolved input types

Closes elastic#3380

Composable packages stack — part 4 of 4 (final).
This PR is stacked on elastic#3329 (closes elastic#3278), #1 (closes elastic#3279), and #2 (closes elastic#3280). Merge order: elastic#3329#1#2 → this PR.
Each PR will be rebased onto main once its predecessor merges.

Summary

Final step of the composable-packages build pipeline: replaces all package: <input-pkg-name> references in policy template inputs and data stream stream entries with the actual input type identifier (e.g. logfile) resolved from the required input package's manifest.

This step runs after variable merging (which relies on stream.Package / input.Package to identify entries to process), so ordering matters.

After this transform the resulting manifests are indistinguishable from hand-authored integration package manifests — the package: key is removed entirely.

What changed

internal/requiredinputs/streamdefs.go (new, ~190 lines)

  • ReplaceStreamPackageRefs: resolves input type from required input package metadata and rewrites both policy_templates[].inputs[] and data_stream/*/manifest.yml → streams[] entries.
  • Falls back to input package title/description when the integration does not declare its own.

internal/requiredinputs/streamdefs_test.go (new, ~460 lines)

  • Covers: policy template input replacement, data stream stream replacement, title/description fallback, multi-stream packages.

How to test

The existing test/manual_packages/required_inputs/with_input_package_requires fixture exercises this path end-to-end:

cd test/manual_packages/required_inputs/test_input_pkg
elastic-package build

elastic-package stack up --version 9.4.0-SNAPSHOT

cd test/manual_packages/required_inputs/with_input_package_requires
elastic-package build
# Inspect build/: policy_templates[].inputs[].type and data_stream streams[].type should now
# contain the resolved input type (e.g. "logfile"); no "package:" key should remain.

Dependencies

teresaromero and others added 2 commits March 26, 2026 16:32
Resolve the final consolidation step for composable packages: replace
all `package: <pkg-name>` references in policy_templates[].inputs[] and
data_stream/*/manifest.yml streams[] with the actual input type
identifier from the referenced input package, then remove the `package:`
key so the result is indistinguishable from a regular integration.

Closes elastic#3380

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Document package: → input type resolution on Resolver and Bundle; expand
build/install docs (README via cmd/build, dependency_management, manual
packages README) for templates, variable merge, fields, and stream types.
Update streamdefs tests to call Bundle.

Made-with: Cursor
@teresaromero teresaromero force-pushed the 3380-replace-stream-packages branch from edf0317 to ed032e8 Compare March 26, 2026 15:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant