Skip to content

feat: bundle field definitions from required input packages#2

Draft
teresaromero wants to merge 3 commits into3279-merge-variables-when-required-inputfrom
3280-bundle-fields
Draft

feat: bundle field definitions from required input packages#2
teresaromero wants to merge 3 commits into3279-merge-variables-when-required-inputfrom
3280-bundle-fields

Conversation

@teresaromero
Copy link
Copy Markdown
Owner

@teresaromero teresaromero commented Mar 24, 2026

Bundle field definitions from required input packages

Closes elastic#3280

Composable packages stack — part 3 of 4.
This PR is stacked on elastic#3329 (bundle templates, closes elastic#3278) and #1 (merge variables, closes elastic#3279). Merge order: elastic#3329#1 → this PR → #3.
Each PR will be rebased onto main once its predecessor merges.

Summary

Extends the required-inputs build step to bundle field definitions from required input packages into the composable integration's data streams.

Bundling rules:

  • Fields already defined in the integration take precedence; only absent fields are copied from the input package.
  • Bundled fields are written to a new file <inputPkgName>-fields.yml inside the data stream's fields/ directory, keeping integration-owned field files untouched.

What changed

internal/requiredinputs/fields.go (new, ~200 lines)

  • BundleFields: entry point called as part of the required-inputs build step.
  • Reads field definitions from the input package, compares against integration-defined fields, and writes missing ones.

internal/requiredinputs/fields_test.go (new, ~300 lines)

  • Covers: full field bundle, partial override (integration fields win), no-op when all fields present.

test/manual_packages/required_inputs/ (new fixtures)

  • fields_input_pkg — input package with a set of base field definitions.
  • with_field_bundling — integration that declares a subset of fields, expecting the rest to be bundled in.

How to test

cd test/manual_packages/required_inputs/fields_input_pkg
elastic-package build

elastic-package stack up --version 9.4.0-SNAPSHOT

cd test/manual_packages/required_inputs/with_field_bundling
elastic-package build
# Inspect data_stream/*/fields/ — should contain <inputPkgName>-fields.yml with bundled fields

Dependencies

teresaromero and others added 2 commits March 26, 2026 16:17
…ifests

Implements variable merging as part of the required_inputs build step
(issue elastic#3279). Input package vars are the base; composable package
overrides win when explicitly specified. Promoted input-level vars and
data-stream-level vars are correctly placed in the merged manifest.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When building composable integration packages, field definitions from
required input packages are now bundled into the integration's data
stream fields directories. Fields already defined in the integration
take precedence; only fields absent from the integration are copied
from the input package into a new file named
<inputPkgName>-fields.yml.

Closes elastic#3280

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove duplicate mergeVariables call in Bundle. Document Resolver, Bundle,
RequiredInputsResolver, and NewRequiredInputsResolver to match the full
pipeline (templates, variables, data stream fields). Broaden the no-requires
debug log. Use Bundle in fields_test.

Made-with: Cursor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant