Add more generic support for var_groups by Supplementing · Pull Request #1120 · elastic/package-spec

Supplementing · 2026-03-23T21:11:03Z

Closes https://github.com/elastic/ingest-dev/issues/7186

What does this PR do?

This PR adds more generic support for usage of var_groups. Previously, we only allowed them at the root and streams level, this PR allows them to be used elsewhere.

Why is it important?

In order to support better organization in the UI of items based on grouping, being able to use the var_groups was paramount. See https://github.com/elastic/ingest-dev/issues/6979

Checklist

I have added test packages to test/packages that prove my change is effective.
I have added an entry in spec/changelog.yml.

coderabbitai · 2026-03-23T21:13:47Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 762ee316-e6be-4695-a0be-8ce08f3b1a5b

📥 Commits

Reviewing files that changed from the base of the PR and between c7cec12 and d6dd7f3.

📒 Files selected for processing (2)

code/go/pkg/validator/validator_test.go
spec/changelog.yml

✅ Files skipped from review due to trivial changes (2)

code/go/pkg/validator/validator_test.go
spec/changelog.yml

📝 Walkthrough

Walkthrough

Schema support for var_groups was added to integration and input package manifests: new var_groups properties at package root, policy template, and input levels, plus migration patches removing them for versions before 3.6.0. A new changelog entry version: 3.6.1-next was added. Tests were extended (validator test) and a new test package good_var_groups_input was added (manifest, templates, fields, docs, changelog, license). The existing good_var_groups test package was updated with additional vars and var_groups. .DS_Store was added to .gitignore.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

🛠️ Update Documentation: Commit on current branch
🛠️ Update Documentation: Create PR

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

spec/integration/manifest.spec.yml (1)

977-984: ⚠️ Potential issue | 🟠 Major

Reorder var_groups removals in the before: 3.6.0 patch.

/definitions/var_groups is removed before properties that still reference it. If patch sequencing enforces schema integrity, this can break backward-compat patching. Remove all .../var_groups properties first, then remove the definition.

Suggested fix

       # Switchable variable groups.
-      - op: remove
-        path: "/definitions/var_groups"
       - op: remove
         path: "/properties/var_groups"
       - op: remove
         path: "/properties/policy_templates/items/properties/var_groups"
       - op: remove
         path: "/properties/policy_templates/items/properties/inputs/items/properties/var_groups"
+      - op: remove
+        path: "/definitions/var_groups"
       - op: remove
         path: "/properties/policy_templates/items/properties/inputs/items/properties/hide_in_var_group_options"

As per coding guidelines: “In version patches, remove property references before removing the definitions they depend on to maintain schema integrity.”

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@spec/integration/manifest.spec.yml` around lines 977 - 984, Reorder the three
remove ops that target property paths so they run before the remove op for the
"/definitions/var_groups" definition: move the ops for "/properties/var_groups",
"/properties/policy_templates/items/properties/var_groups", and
"/properties/policy_templates/items/properties/inputs/items/properties/var_groups"
ahead of the op for "/definitions/var_groups" so all references to var_groups
are removed before the definition itself (keep the same op entries but change
their sequence).

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@spec/changelog.yml`:
- Around line 27-29: The changelog entry in spec/changelog.yml currently uses a
link value of "https://github.com/elastic/package-spec/pull/PLACEHOLDER"; update
the link field to follow changelog conventions by replacing "PLACEHOLDER" with
"TBD" or the real PR URL ("https://github.com/elastic/package-spec/pull/1120")
so the YAML entry under description/type/link reflects the correct link format.

---

Outside diff comments:
In `@spec/integration/manifest.spec.yml`:
- Around line 977-984: Reorder the three remove ops that target property paths
so they run before the remove op for the "/definitions/var_groups" definition:
move the ops for "/properties/var_groups",
"/properties/policy_templates/items/properties/var_groups", and
"/properties/policy_templates/items/properties/inputs/items/properties/var_groups"
ahead of the op for "/definitions/var_groups" so all references to var_groups
are removed before the definition itself (keep the same op entries but change
their sequence).

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: e2d6dc18-1524-492d-b783-bf68947e9bce

📥 Commits

Reviewing files that changed from the base of the PR and between 3d8508e and 8ee00a9.

⛔ Files ignored due to path filters (8)

.DS_Store is excluded by !**/.DS_Store
.buildkite/.DS_Store is excluded by !**/.DS_Store
.github/.DS_Store is excluded by !**/.DS_Store
code/.DS_Store is excluded by !**/.DS_Store
compliance/.DS_Store is excluded by !**/.DS_Store
spec/.DS_Store is excluded by !**/.DS_Store
test/.DS_Store is excluded by !**/.DS_Store
test/packages/good_var_groups_input/img/sample-logo.svg is excluded by !**/*.svg

📒 Files selected for processing (11)

code/go/pkg/validator/validator_test.go
spec/changelog.yml
spec/input/manifest.spec.yml
spec/integration/manifest.spec.yml
test/packages/good_var_groups/manifest.yml
test/packages/good_var_groups_input/LICENSE.txt
test/packages/good_var_groups_input/agent/input/input.yml.hbs
test/packages/good_var_groups_input/changelog.yml
test/packages/good_var_groups_input/docs/README.md
test/packages/good_var_groups_input/fields/base-fields.yml
test/packages/good_var_groups_input/manifest.yml

## Summary Closes elastic/ingest-dev#7185 Related: elastic/ingest-dev#6549 Depends on elastic/package-spec#1120 This PR makes `var_groups` usable everywhere, rather than just at the package root and streams level so that items can be rendered in their appropriate sections (input, etc) and take full advantage of grouping. This will unblock elastic/integrations#17495 and allow us to finalize a draft version of this integration as a poster-child that utilizes `var_groups` for a clean and minimal UI. WIP integration utilizing the var_groups at the input level for grouping based on chosen auth method: <img width="1709" height="1198" alt="image" src="https://github.com/user-attachments/assets/433307f4-dcd6-460f-a34f-2a5fa9dc6c3f" /> ## Testing instructions If you want to manually build an integration and test, please DM me and I will send you detailed instructions or walk you through how to set up a local package-spec and point your elastic-package at it, build it, etc. Im happy to do so, but in the sake of brevity, I will skip that here. The easiest way otherwise is to simply upload this pre-built integration that I built locally using the changes to package-spec: [okta-3.14.2.zip](https://github.com/user-attachments/files/26195499/okta-3.14.2.zip) ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [ ] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

## Summary Closes elastic/ingest-dev#7185 Related: elastic/ingest-dev#6549 Depends on elastic/package-spec#1120 This PR makes `var_groups` usable everywhere, rather than just at the package root and streams level so that items can be rendered in their appropriate sections (input, etc) and take full advantage of grouping. This will unblock elastic/integrations#17495 and allow us to finalize a draft version of this integration as a poster-child that utilizes `var_groups` for a clean and minimal UI. WIP integration utilizing the var_groups at the input level for grouping based on chosen auth method: <img width="1709" height="1198" alt="image" src="https://github.com/user-attachments/assets/433307f4-dcd6-460f-a34f-2a5fa9dc6c3f" /> ## Testing instructions If you want to manually build an integration and test, please DM me and I will send you detailed instructions or walk you through how to set up a local package-spec and point your elastic-package at it, build it, etc. Im happy to do so, but in the sake of brevity, I will skip that here. The easiest way otherwise is to simply upload this pre-built integration that I built locally using the changes to package-spec: [okta-3.14.2.zip](https://github.com/user-attachments/files/26195499/okta-3.14.2.zip) ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [ ] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

mrodm

LGTM
Since 3.6.0 has been released, the changelog must be updated to include this entry into a new 3.6.1-next section.

…erywhere

elasticmachine · 2026-03-31T15:28:26Z

💚 Build Succeeded

Buildkite Build
Commit: d6dd7f3

History

💚 Build #1996 succeeded c7cec12
💚 Build #1995 succeeded 2365b69
💚 Build #1989 succeeded 696a450
💚 Build #1985 succeeded 8ee00a9

add support for var groups at other levels

85435e9

Supplementing requested a review from a team as a code owner March 23, 2026 21:11

Supplementing added the Team:Fleet Label for the Fleet team label Mar 23, 2026

added test package

8ee00a9

coderabbitai Bot reviewed Mar 23, 2026

View reviewed changes

Comment thread spec/changelog.yml Outdated

Supplementing mentioned this pull request Mar 23, 2026

[Fleet] Make var groups more generically usable elastic/kibana#259213

Merged

10 tasks

fixed placeholder and changed ordering of removal per coderabbit comment

696a450

teresaromero reviewed Mar 25, 2026

View reviewed changes

Comment thread .buildkite/.DS_Store Outdated

Remove accidental .DS_Store files from repository

2365b69

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Supplementing requested a review from teresaromero March 25, 2026 16:07

mrodm reviewed Mar 25, 2026

View reviewed changes

Comment thread spec/input/manifest.spec.yml

Comment thread spec/integration/manifest.spec.yml

added ds store to git ignore

c7cec12

Supplementing requested a review from mrodm March 25, 2026 18:51

mrodm reviewed Mar 31, 2026

View reviewed changes

Comment thread spec/changelog.yml Outdated

Comment thread spec/input/manifest.spec.yml

Comment thread .buildkite/.DS_Store Outdated

Supplementing added 2 commits March 31, 2026 09:06

Merge remote-tracking branch 'upstream' into add-var-group-support-ev…

f2383de

…erywhere

fixed changelog

d6dd7f3

Supplementing requested a review from mrodm March 31, 2026 15:11

mrodm approved these changes Mar 31, 2026

View reviewed changes

mrodm merged commit 1e6687f into elastic:main Mar 31, 2026
5 checks passed

teresaromero mentioned this pull request Apr 21, 2026

Prepare release 3.6.1 #1145

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add more generic support for var_groups#1120

Add more generic support for var_groups#1120
mrodm merged 7 commits intoelastic:mainfrom
Supplementing:add-var-group-support-everywhere

Supplementing commented Mar 23, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Mar 23, 2026 •

edited

Loading

Walkthrough

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mrodm left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elasticmachine commented Mar 31, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Supplementing commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Why is it important?

Checklist

Uh oh!

coderabbitai Bot commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mrodm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elasticmachine commented Mar 31, 2026

💚 Build Succeeded

History

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Supplementing commented Mar 23, 2026 •

edited

Loading

coderabbitai Bot commented Mar 23, 2026 •

edited

Loading