[Star-tree] Support for nested aggs & Removing experimental flag. #10132

sandeshkr419 · 2025-06-23T18:22:31Z

Description

Nested Aggs was added and experimental feature flag was removed in 3.1 for star-tree:

opensearch-project/OpenSearch#18048
opensearch-project/OpenSearch#18070

Issues Resolved

Closes #[delete this text, including the brackets, and replace with the issue number]

Version

3.1

Frontend features

If you're submitting documentation for an OpenSearch Dashboards feature, add a video that shows how a user will interact with the UI step by step. A voiceover is optional.

Checklist

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

nested aggs support Signed-off-by: Sandesh Kumar <[email protected]>

Signed-off-by: Sandesh Kumar <[email protected]>

github-actions · 2025-06-23T18:22:37Z

Thank you for submitting your PR. The PR states are In progress (or Draft) -> Tech review -> Doc review -> Editorial review -> Merged.

Before you submit your PR for doc review, make sure the content is technically accurate. If you need help finding a tech reviewer, tag a maintainer.

When you're ready for doc review, tag the assignee of this PR. The doc reviewer may push edits to the PR directly or leave comments and editorial suggestions for you to address (let us know in a comment if you have a preference). The doc reviewer will arrange for an editorial review.

Signed-off-by: Sandesh Kumar <[email protected]>

Signed-off-by: Fanit Kolchina <[email protected]>

_search-plugins/star-tree-index.md

sandeshkr419 · 2025-06-23T21:14:56Z

_search-plugins/star-tree-index.md

-## Using queries without a star-tree index
+#### Nested aggregations

-Set the `indices.composite_index.star_tree.enabled` setting to `false` to run queries without using a star-tree index.


@kolchfa-aws We should not remove this section. We need to still give the user an option to disable index search via star-tree.

I'll incorporate it into the "enabling" section.

Can we kindly keep this a separate section. Today with feature flag removal, and cluster setting pointing to true, its by default enabled. So its not coming out clearly that this can be done. Also we have added index level setting so adding the info in separate section will be useful.

kolchfa-aws · 2025-06-23T21:20:09Z

_search-plugins/star-tree-index.md


- Set the feature flag `opensearch.experimental.feature.composite_index.star_tree.enabled` to `true`. For more information about enabling and disabling feature flags, see [Enabling experimental features]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/experimental/).
- Set the `indices.composite_index.star_tree.enabled` setting to `true`. For instructions on how to configure OpenSearch, see [Configuring settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/#static-settings).
+- Set the `indices.composite_index.star_tree.enabled` setting to `true`. For more information, see [Dynamic settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/#dynamic-settings).


Suggested change

- Set the `indices.composite_index.star_tree.enabled` setting to `true`. For more information, see [Dynamic settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/#dynamic-settings).

Signed-off-by: kolchfa-aws <[email protected]>

sandeshkr419 · 2025-06-23T21:25:33Z

_search-plugins/star-tree-index.md

+
+- Enable a star-tree index only for indexes whose data is not updated or deleted: star-tree indexes do not support updates or deletions. To enforce this policy, set `index.append_only.enabled` to `true` for the index you want to enable star-tree index on.
+- Use a star-tree index for aggregation queries only if the queried fields are a subset of the star-tree's dimensions and the aggregated fields are a subset of the star-tree's metrics.
+- Once enabled, a star-tree index cannot be disabled. In order to disable a star-tree index, the data in the index must be reindexed without the star-tree mapping. Changing a star-tree configuration also requires reindexing data.


Regarding 6848b61#r2162545785.

I think this is a good place to include this information. Something like "Searches using star-tree can however be disabled by this setting.

Basically setting indices.composite_index.star_tree.enabled to false will still create index structures related to star-tree however, the searches will be directed to default path. I'm not sure how to frame this information in a ncie way. @kolchfa-aws Please feel free to reword this section.

Thanks! Reworded.

Signed-off-by: Fanit Kolchina <[email protected]>

sandeshkr419 · 2025-06-23T21:44:17Z

Thanks @kolchfa-aws for rewording the pages. Looks neat now. Please move to editorial section.

Signed-off-by: Fanit Kolchina <[email protected]>

kolchfa-aws

Thank you, @sandeshkr419! Moving on to editorial review.

expani · 2025-06-24T00:55:35Z

_search-plugins/star-tree-index.md


-This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, join the discussion on the [OpenSearch forum](https://forum.opensearch.org/).    
-{: .warning}
+A _star-tree index_ is a multi-field index that improves the performance of aggregations by precomputing metric values for combinations of dimension fields.


nit: _star-tree

Also for index_

Its putting the words in italic font : _italic font_

_search-plugins/star-tree-index.md

expani · 2025-06-24T03:59:34Z

_search-plugins/star-tree-index.md

- Star-tree indexes are created in real time as part of the indexing process, so the data in a star-tree will always be up to date.
- A star-tree index consolidates data, increasing index paging efficiency and using less IO for search queries.
+- Star-tree indexes are created in real time as part of the indexing process, so the data in a star-tree is always current.
+- A star-tree index consolidates data to improve paging efficiency and reduce disk I/O during search queries.


Unrelated to this PR.

What does Consolidates mean here ? The aggregation/pre-cubing from raw documents to create StarTreeDocuments that are unique ?

_search-plugins/star-tree-index.md

Signed-off-by: Fanit Kolchina <[email protected]>

natebower

Editorial review

_search-plugins/star-tree-index.md

Signed-off-by: Nathan Bower <[email protected]>

Signed-off-by: Sandesh Kumar <[email protected]>

sandeshkr419

@kolchfa-aws I have addressed the technical comments by @bharath-techie & @expani.

Please take a look now.

_search-plugins/star-tree-index.md

sandeshkr419 · 2025-06-24T19:02:53Z

_search-plugins/star-tree-index.md

-## Using queries without a star-tree index
+#### Nested aggregations

-Set the `indices.composite_index.star_tree.enabled` setting to `false` to run queries without using a star-tree index.


Signed-off-by: Fanit Kolchina <[email protected]>

natebower · 2025-06-24T19:49:30Z

_search-plugins/star-tree-index.md

+| `indices.composite_index.star_tree.enabled` | Cluster | `true`  | Enables or disables star-tree search optimization across the cluster.  |
+| `index.composite_index`                     | Index   | None       | Enables star-tree indexing for a specific index. Must be set when creating the index.                                                |
+| `index.append_only.enabled`                 | Index   | None      | Required for star-tree indexes. Prevents updates and deletions. Must be `true`.                                                      |
+| `index.search.star_tree_index.enabled`      | Index   | `true`  | Enables or disables use of the star-tree index for search queries on this index.                                                 |


"this" => "the"

_search-plugins/star-tree-index.md

Signed-off-by: kolchfa-aws <[email protected]>

_search-plugins/star-tree-index.md

Signed-off-by: kolchfa-aws <[email protected]>

…ensearch-project#10132) * Update star-tree-index.md nested aggs support Signed-off-by: Sandesh Kumar <[email protected]> * remove experimental banner Signed-off-by: Sandesh Kumar <[email protected]> * title capital letter error Signed-off-by: Sandesh Kumar <[email protected]> * remove experimental flag setting Signed-off-by: Sandesh Kumar <[email protected]> * Doc review Signed-off-by: Fanit Kolchina <[email protected]> * Update _search-plugins/star-tree-index.md Signed-off-by: kolchfa-aws <[email protected]> * Incorporate disabling star-tree in enabling section Signed-off-by: Fanit Kolchina <[email protected]> * Restructure limitations Signed-off-by: Fanit Kolchina <[email protected]> * Rewrite enabling section Signed-off-by: Fanit Kolchina <[email protected]> * More rewording Signed-off-by: Fanit Kolchina <[email protected]> * Add cross-links Signed-off-by: Fanit Kolchina <[email protected]> * Minor rewording Signed-off-by: Fanit Kolchina <[email protected]> * Apply suggestions from code review Signed-off-by: Nathan Bower <[email protected]> * Address technical comments Signed-off-by: Sandesh Kumar <[email protected]> * doc review Signed-off-by: Fanit Kolchina <[email protected]> * Update _search-plugins/star-tree-index.md Signed-off-by: kolchfa-aws <[email protected]> * Update _search-plugins/star-tree-index.md Signed-off-by: kolchfa-aws <[email protected]> --------- Signed-off-by: Sandesh Kumar <[email protected]> Signed-off-by: Fanit Kolchina <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> Signed-off-by: Nathan Bower <[email protected]> Co-authored-by: Fanit Kolchina <[email protected]> Co-authored-by: kolchfa-aws <[email protected]> Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: apple <[email protected]>

…ensearch-project#10132) * Update star-tree-index.md nested aggs support Signed-off-by: Sandesh Kumar <[email protected]> * remove experimental banner Signed-off-by: Sandesh Kumar <[email protected]> * title capital letter error Signed-off-by: Sandesh Kumar <[email protected]> * remove experimental flag setting Signed-off-by: Sandesh Kumar <[email protected]> * Doc review Signed-off-by: Fanit Kolchina <[email protected]> * Update _search-plugins/star-tree-index.md Signed-off-by: kolchfa-aws <[email protected]> * Incorporate disabling star-tree in enabling section Signed-off-by: Fanit Kolchina <[email protected]> * Restructure limitations Signed-off-by: Fanit Kolchina <[email protected]> * Rewrite enabling section Signed-off-by: Fanit Kolchina <[email protected]> * More rewording Signed-off-by: Fanit Kolchina <[email protected]> * Add cross-links Signed-off-by: Fanit Kolchina <[email protected]> * Minor rewording Signed-off-by: Fanit Kolchina <[email protected]> * Apply suggestions from code review Signed-off-by: Nathan Bower <[email protected]> * Address technical comments Signed-off-by: Sandesh Kumar <[email protected]> * doc review Signed-off-by: Fanit Kolchina <[email protected]> * Update _search-plugins/star-tree-index.md Signed-off-by: kolchfa-aws <[email protected]> * Update _search-plugins/star-tree-index.md Signed-off-by: kolchfa-aws <[email protected]> --------- Signed-off-by: Sandesh Kumar <[email protected]> Signed-off-by: Fanit Kolchina <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> Signed-off-by: Nathan Bower <[email protected]> Co-authored-by: Fanit Kolchina <[email protected]> Co-authored-by: kolchfa-aws <[email protected]> Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Eric Pugh <[email protected]>

sandeshkr419 added 2 commits June 23, 2025 11:19

Update star-tree-index.md

347e6f3

nested aggs support Signed-off-by: Sandesh Kumar <[email protected]>

remove experimental banner

53e2b71

Signed-off-by: Sandesh Kumar <[email protected]>

sandeshkr419 requested review from AMoo-Miki, Naarcha-AWS, dlvenable, epugh, kolchfa-aws and natebower as code owners June 23, 2025 18:22

github-actions bot assigned kolchfa-aws Jun 23, 2025

Merge branch 'main' into patch-1

46ad312

sandeshkr419 added the v3.1.0 label Jun 23, 2025

title capital letter error

741f811

Signed-off-by: Sandesh Kumar <[email protected]>

sandeshkr419 changed the title ~~Support for nested aggs & Removing experimental flag.~~ [Star-tree] Support for nested aggs & Removing experimental flag. Jun 23, 2025

sandeshkr419 and others added 3 commits June 23, 2025 13:23

remove experimental flag setting

50dbfe2

Signed-off-by: Sandesh Kumar <[email protected]>

Doc review

6848b61

Signed-off-by: Fanit Kolchina <[email protected]>

Resolve merge conflicts

2b32454

Signed-off-by: Fanit Kolchina <[email protected]>

kolchfa-aws reviewed Jun 23, 2025

View reviewed changes

_search-plugins/star-tree-index.md Show resolved Hide resolved

kolchfa-aws reviewed Jun 23, 2025

View reviewed changes

_search-plugins/star-tree-index.md Outdated Show resolved Hide resolved

sandeshkr419 commented Jun 23, 2025

View reviewed changes

kolchfa-aws reviewed Jun 23, 2025

View reviewed changes

Update _search-plugins/star-tree-index.md

e8c9292

Signed-off-by: kolchfa-aws <[email protected]>

sandeshkr419 commented Jun 23, 2025

View reviewed changes

Incorporate disabling star-tree in enabling section

c28f6f7

Signed-off-by: Fanit Kolchina <[email protected]>

Restructure limitations

ee8d3c9

Signed-off-by: Fanit Kolchina <[email protected]>

kolchfa-aws added the Editorial review PR: Editorial review in progress label Jun 23, 2025

kolchfa-aws assigned natebower Jun 23, 2025

kolchfa-aws approved these changes Jun 23, 2025

View reviewed changes

expani reviewed Jun 24, 2025

View reviewed changes

bharath-techie reviewed Jun 24, 2025

View reviewed changes

_search-plugins/star-tree-index.md Outdated Show resolved Hide resolved

bharath-techie reviewed Jun 24, 2025

View reviewed changes

_search-plugins/star-tree-index.md Outdated Show resolved Hide resolved

bharath-techie reviewed Jun 24, 2025

View reviewed changes

_search-plugins/star-tree-index.md Show resolved Hide resolved

expani reviewed Jun 24, 2025

View reviewed changes

_search-plugins/star-tree-index.md Outdated Show resolved Hide resolved

expani reviewed Jun 24, 2025

View reviewed changes

natebower added Tech review PR: Tech review in progress and removed Editorial review PR: Editorial review in progress labels Jun 24, 2025

natebower removed their assignment Jun 24, 2025

kolchfa-aws added 4 commits June 24, 2025 09:22

Rewrite enabling section

764d2e9

Signed-off-by: Fanit Kolchina <[email protected]>

More rewording

bf4f301

Signed-off-by: Fanit Kolchina <[email protected]>

Add cross-links

23a0ba2

Signed-off-by: Fanit Kolchina <[email protected]>

Minor rewording

eb55f86

Signed-off-by: Fanit Kolchina <[email protected]>

natebower reviewed Jun 24, 2025

View reviewed changes

Apply suggestions from code review

1e8b57c

Signed-off-by: Nathan Bower <[email protected]>

kolchfa-aws added the release-notes PR: Include this PR in the automated release notes label Jun 24, 2025

Address technical comments

788d8b7

Signed-off-by: Sandesh Kumar <[email protected]>

sandeshkr419 commented Jun 24, 2025

View reviewed changes

sandeshkr419 and others added 2 commits June 24, 2025 12:05

Merge branch 'main' into patch-1

837516b

doc review

428b83b

Signed-off-by: Fanit Kolchina <[email protected]>

natebower reviewed Jun 24, 2025

View reviewed changes

kolchfa-aws reviewed Jun 24, 2025

View reviewed changes

_search-plugins/star-tree-index.md Outdated Show resolved Hide resolved

Update _search-plugins/star-tree-index.md

996322f

Signed-off-by: kolchfa-aws <[email protected]>

kolchfa-aws reviewed Jun 24, 2025

View reviewed changes

_search-plugins/star-tree-index.md Outdated Show resolved Hide resolved

Update _search-plugins/star-tree-index.md

fd84526

Signed-off-by: kolchfa-aws <[email protected]>

kolchfa-aws merged commit 4e26f74 into opensearch-project:main Jun 24, 2025
5 checks passed

sandeshkr419 deleted the patch-1 branch June 24, 2025 20:39

[Star-tree] Support for nested aggs & Removing experimental flag. #10132

[Star-tree] Support for nested aggs & Removing experimental flag. #10132

Uh oh!

Conversation

sandeshkr419 commented Jun 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Issues Resolved

Version

Frontend features

Checklist

Uh oh!

github-actions bot commented Jun 23, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sandeshkr419 commented Jun 23, 2025

Uh oh!

kolchfa-aws left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

natebower left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sandeshkr419 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

sandeshkr419 commented Jun 23, 2025 •

edited

Loading