Support window functions in time-series aggregations #138139

dnhatn · 2025-11-16T18:44:53Z

This change adds window function support for time-series aggregation functions including rate, avg_over_time, and last_over_time. Additional time-series aggregation functions will be supported in a follow-up to keep this PR focused.

Examples:

TS k8s
| WHERE TRANGE("2024-05-10T00:05:00.000Z", "2024-05-10T00:10:00.000Z")
| STATS rate_bytes_in=avg(rate(network.total_bytes_in, 5minute)) 
              BY cluster, time_bucket = bucket(@timestamp, 1minute)

Window 5-minute over 1-minute buckets.

TS k8s 
| STATS events = sum(avg_over_time(events_received, 10minute)) 
             BY pod, time_bucket = tbucket(5minute)
| SORT time_bucket, pod
| LIMIT 10

Window 10-minute over 5-minute buckets.

Limitation:

ES|QL currently supports only windows that are multiples of the bucket size.

elasticsearchmachine · 2025-11-16T18:45:40Z

Hi @dnhatn, I've created a changelog YAML for you.

github-actions · 2025-11-16T18:47:17Z

ℹ️ Important: Docs version tagging

👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version.

We use applies_to tags to mark version-specific features and changes.

Expand for a quick overview

When to use applies_to tags:

✅ At the page level to indicate which products/deployments the content applies to (mandatory)
✅ When features change state (e.g. preview, ga) in a specific version
✅ When availability differs across deployments and environments

What NOT to do:

❌ Don't remove or replace information that applies to an older version
❌ Don't add new information that applies to a specific version without an applies_to tag
❌ Don't forget that applies_to tags can be used at the page, section, and inline level

🤔 Need help?

Check out the cumulative docs guidelines
Reach out in the #docs Slack channel

dnhatn · 2025-11-16T19:13:14Z

...src/main/java/org/elasticsearch/compute/aggregation/X-RateGroupingAggregatorFunction.java.st

                    Interval prev = intervals[i];
-                    if (prev.v2 > next.v2) {
-                        state.resets += prev.v2;
+                    if (prev.v1 > next.v2) {


Existing bug in rate that only manifests when using window functions.

elasticsearchmachine · 2025-11-17T00:17:11Z

Pinging @elastic/es-storage-engine (Team:StorageEngine)

kkrik-es · 2025-11-17T06:31:51Z

...te/src/main/java/org/elasticsearch/compute/aggregation/WindowGroupingAggregatorFunction.java

+        TimeSeriesGroupingAggregatorEvaluationContext evaluationContext
+    ) {
+        if (selected.getPositionCount() > 0) {
+            // TODO: rewrite to NO_WINDOW in the planner if the bucket and the window are the same


What happens if the window is smaller than the bucket, e.g.

TS metrics | STATS sum(rate(reqs, 1m) BY TBUCKET(5m)

We should reject this query during translation.

kkrik-es · 2025-11-17T06:45:20Z

...ql/src/main/java/org/elasticsearch/xpack/esql/expression/function/aggregate/AvgOverTime.java

-        examples = { @Example(file = "k8s-timeseries", tag = "avg_over_time") }
+        examples = {
+            @Example(file = "k8s-timeseries", tag = "avg_over_time"),
+            @Example(file = "k8s-timeseries-avg-over-time", tag = "avg_over_time_with_window") }


Can we skip the version with window for now? I think it's gonna show up in the documentation, which will be confusing for users in 9.2.. Let's discuss separate with Liam on how best to approach this, once we have all functions updated to support windows.

We do need to add them in the tests so that Kibana documentation is updated..

++ I removed this in 3e42bb7

kkrik-es · 2025-11-17T06:52:11Z

...gin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/LogicalPlanOptimizerTests.java

+            int window = between(1, 20);
+            var query = String.format(Locale.ROOT, """
+                TS k8s
+                | STATS avg(last_over_time(network.bytes_in, %s minute)) BY bucket(@timestamp, 1 minute)


Nit: use TBUCKET?

kkrik-es · 2025-11-17T06:54:33Z

.../java/org/elasticsearch/xpack/esql/optimizer/rules/logical/TranslateTimeSeriesAggregate.java

+                    }
+                }
+                throw new EsqlIllegalArgumentException(
+                    "Unsupported window [{}] of aggregate function [{}]; the window must be a positive multiple of the time bucket [{}].",


or smaller than the time bucker?

++ I pushed e0a3c5a

kkrik-es

Nice, so clean! Mostly a question around the documentation, to avoid confusing users of 9.2.

…rate

dnhatn · 2025-11-17T19:28:37Z

Thanks Kostas!

window!

aac9e59

dnhatn added >enhancement :StorageEngine/ES|QL Timeseries / metrics / logsdb capabilities in ES|QL labels Nov 16, 2025

elasticsearchmachine added the v9.3.0 label Nov 16, 2025

Update docs/changelog/138139.yaml

a56b771

dnhatn changed the title ~~Support window function in time-series aggregations~~ Support window functions in time-series aggregations Nov 16, 2025

dnhatn commented Nov 16, 2025

View reviewed changes

v0

79aaf81

dnhatn requested a review from kkrik-es November 17, 2025 00:16

dnhatn marked this pull request as ready for review November 17, 2025 00:16

elasticsearchmachine added the Team:StorageEngine label Nov 17, 2025

kkrik-es reviewed Nov 17, 2025

View reviewed changes

kkrik-es approved these changes Nov 17, 2025

View reviewed changes

dnhatn added 5 commits November 17, 2025 08:56

Remove examples

3e42bb7

better error message

e0a3c5a

Merge remote-tracking branch 'elastic/main' into add-window-function-…

05c45c0

…rate

stylecheck

c8bd00c

stylecheck

a4cd267

dnhatn merged commit 59da11b into elastic:main Nov 17, 2025
36 checks passed

dnhatn deleted the add-window-function-rate branch November 17, 2025 19:28

Support window functions in time-series aggregations #138139

Support window functions in time-series aggregations #138139

Uh oh!

Conversation

dnhatn commented Nov 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticsearchmachine commented Nov 16, 2025

Uh oh!

github-actions bot commented Nov 16, 2025

ℹ️ Important: Docs version tagging

When to use applies_to tags:

What NOT to do:

🤔 Need help?

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elasticsearchmachine commented Nov 17, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kkrik-es left a comment

Choose a reason for hiding this comment

Uh oh!

dnhatn commented Nov 17, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

dnhatn commented Nov 16, 2025 •

edited

Loading