[ESQL] Adds parsing and planner wiring for LIMIT BY by ncordon · Pull Request #143762 · elastic/elasticsearch

ncordon · 2026-03-06T17:40:19Z

This is part of the new LIMIT BY ESQL command. LIMIT N BY expr1, expr2,... retains at most N documents per group, where groups are defined by one or more grouping key expressions.

This PR knits the parser changes and the planner changes for the compute side, GroupedLimitOperator that was added in #143458.

This means it enables queries like this:

FROM test
| LIMIT 5 BY languages
| KEEP first_name, last_name, salary, languages

but we have restricted using a SORT before the LIMIT BY until #143476 has been merged. We have also done this for cleanliness because otherwise this PR would be even bigger. This means queries like this one would fail:

FROM test
| SORT first_name
| LIMIT 5 BY languages
| KEEP first_name, last_name, salary, languages

Part of #112918, https://github.com/elastic/esql-planning/issues/238

coderabbitai · 2026-03-06T17:41:05Z

📝 Walkthrough

Walkthrough

This pull request implements LIMIT BY functionality for ES SQL, enabling limiting results grouped by specified column(s). Changes include grammar updates to parse LIMIT constant BY grouping expressions, logical plan modifications to the Limit class to store groupings, and physical plan updates to LimitExec for execution. New optimizer rules handle expression replacement and literal pruning in LIMIT BY clauses. Local execution planning adds GroupedLimitOperator support. The feature is development-gated via isDevVersion(). Verification logic prevents SORT before LIMIT BY. Comprehensive test coverage includes parser tests, optimizer rule tests, serialization tests, and golden test updates reflecting the new plan structure with empty groupings lists.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

🛠️ Update Documentation: Commit on current branch
🛠️ Update Documentation: Create PR

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ast-grep (0.41.0)

x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/analysis/AnalyzerTests.java

x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/LocalPhysicalPlanOptimizerTests.java

x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/LogicalPlanOptimizerTests.java

2 others

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Copilot

Pull request overview

Adds end-to-end support for the new ESQL LIMIT N BY ... form by extending parsing, logical/physical planning, optimization, and local execution to carry grouping keys and execute a grouped-limit operator, with BWC-aware serialization and new test coverage.

Changes:

Extend LIMIT/LimitExec to carry groupings (for LIMIT BY) including transport-version-gated serialization.
Add logical optimization to (a) prune foldable BY keys and (b) rewrite non-attribute BY expressions into a synthetic EVAL + PROJECT while preserving output.
Wire physical/local planning to execute grouped limits via GroupedLimitOperator, and prevent pushing LIMIT BY into the source.

Reviewed changes

Copilot reviewed 33 out of 34 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/plan/physical/LimitExecSerializationTests.java	Updates serialization tests for `LimitExec` groupings and adds BWC tests for old transport versions.
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/plan/logical/LimitSerializationTests.java	Updates logical `Limit` serialization tests and adds BWC assertions around `LIMIT BY`.
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/parser/StatementParserTests.java	Adds parser coverage for `LIMIT ... BY ...` (including qualified names).
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/ReplaceLimitByExpressionWithEvalTests.java	New tests for pruning foldable `BY` keys and extracting expression keys into `EVAL` + output-preserving `PROJECT`.
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/PhysicalPlanOptimizerTests.java	Adds physical optimizer assertions around `LIMIT BY` behavior (no pushdown to source, eval insertion expectations).
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/LogicalPlanOptimizerTests.java	Adds logical optimizer assertions for `LIMIT BY` interactions/combination behavior.
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/LocalPhysicalPlanOptimizerTests.java	Adds local planner/optimizer assertions around `LIMIT BY` not pushing down to `EsQueryExec`.
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/analysis/VerifierTests.java	Adds verifier tests for disallowing `SORT` before `LIMIT BY` and unknown grouping columns.
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/analysis/AnalyzerTests.java	Ensures implicit default limit behavior works with `LIMIT BY`.
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/planner/mapper/Mapper.java	Wires logical `Limit.groupings()` into `LimitExec` in distributed mapper.
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/planner/mapper/LocalMapper.java	Wires logical `Limit.groupings()` into `LimitExec` in local mapper.
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/planner/LocalExecutionPlanner.java	Plans `LIMIT BY` as `GroupedLimitOperator` (and plain `LIMIT` as `LimitOperator`).
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/plan/physical/LimitExec.java	Adds `groupings` to physical plan node + transport-version-aware serialization.
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/plan/logical/Limit.java	Adds `groupings` to logical plan node + transport-version-aware serialization and resolution.
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/parser/LogicalPlanBuilder.java	Parses `LIMIT N BY ...` and populates logical `Limit` with grouping expressions.
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/parser/EsqlBaseParserVisitor.java	Regenerated visitor interface to include `limitByGroupKey`.
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/parser/EsqlBaseParserListener.java	Regenerated listener interface to include `limitByGroupKey`.
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/parser/EsqlBaseParserBaseVisitor.java	Regenerated base visitor to include `visitLimitByGroupKey`.
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/parser/EsqlBaseParserBaseListener.java	Regenerated base listener to include `enter/exitLimitByGroupKey`.
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/parser/EsqlBaseParser.interp	Regenerated parser artifacts reflecting the new grammar rule.
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/PushLimitToSource.java	Prevents pushing `LIMIT BY` into the source query.
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/ReplaceLimitByExpressionWithEval.java	New rule to prune foldable group keys and rewrite expression keys into `EVAL` + output-preserving `PROJECT`.
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/PushDownAndCombineLimits.java	Updates limit pushdown/combination logic to respect grouping keys.
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/PruneColumns.java	Ensures grouped `Limit` participates in reference tracking (cannot be treated as pass-through).
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/LogicalPlanOptimizer.java	Registers `ReplaceLimitByExpressionWithEval` in the substitutions batch.
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/core/expression/Expressions.java	Adds `listSemanticEquals` helper for comparing grouping lists semantically.
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/analysis/Verifier.java	Adds verification that `SORT` cannot be used before `LIMIT BY` (current restriction).
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/EsqlCapabilities.java	Adds `LIMIT_BY` capability flag.
x-pack/plugin/esql/src/main/antlr/EsqlBaseParser.g4	Extends `limitCommand` grammar to optionally accept `BY fields` (dev-gated).
x-pack/plugin/esql/qa/testFixtures/src/main/resources/topN.csv-spec	Adds QA CSV specs exercising `LIMIT BY` scenarios (multi-keys, expressions, nulls, limit 0, etc.).
x-pack/plugin/esql/build.gradle	Tweaks `regenParser` args for parser regeneration.
server/src/main/resources/transport/upper_bounds/9.4.csv	Advances transport version upper bound to `esql_limit_by`.
server/src/main/resources/transport/definitions/referable/esql_limit_by.csv	Introduces transport-version definition for `esql_limit_by`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-09T18:48:56Z

x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/EsqlCapabilities.java

+        /**
+
+         * Enables LIMIT N BY in the LIMIT command, both with and without a preceding SORT.
+         */
+        LIMIT_BY,


The Javadoc for LIMIT_BY says it enables LIMIT N BY "both with and without a preceding SORT", but Verifier#checkLimitBy currently rejects SORT before LIMIT BY. Please update this comment to reflect the current restriction (or relax the verifier if the intent is to support SORT-before-LIMIT BY now).

This was addressed

elasticsearchmachine · 2026-03-10T09:12:22Z

Pinging @elastic/es-analytical-engine (Team:Analytics)

julian-elastic · 2026-03-10T12:51:12Z

x-pack/plugin/esql/qa/testFixtures/src/main/resources/topN.csv-spec

+
+FROM employees
+| WHERE emp_no IN (10001, 10002, 10003, 10005, 10006)
+| LIMIT 2 BY 5*42


How is this stable? This UT could start failing due to different results every time you run it?

Good point! I've made this stable now (I think)

julian-elastic · 2026-03-10T12:53:53Z

x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/analysis/Verifier.java


+    private static void checkLimitBy(LogicalPlan plan, Failures failures) {
+        if (plan instanceof Limit limit && limit.groupings().isEmpty() == false) {
+            if (limit.child() instanceof OrderBy) {


Are you only checking the immediate child here? What if there is a sort further down the plan?

Good point! I've added a check for intermediate children now. We should give a verification error if we have a LIMIT BY + some other commands excluding a LIMIT without groupings + SORT

x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/plan/logical/Limit.java

julian-elastic

Overall looks good! I left some small comments you need to address before check in, but nothing major.

ivancea

First pass over the non-test code

ivancea · 2026-03-10T11:55:04Z

x-pack/plugin/esql/qa/testFixtures/src/main/resources/topN.csv-spec

These tests aren't TopN related, so I would move them to limit.csv-spec (Which has a single test, so a good candidate!)

ivancea · 2026-03-10T12:10:01Z

...k/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/LogicalPlanOptimizer.java

            new ReplaceOrderByExpressionWithEval(),
+            new ReplaceLimitByExpressionWithEval(),


I see ReplaceLimitByExpressionWithEval also prune literals, which is something ORDER BY does in PruneLiteralsInOrderBy.
Given the similarity, I would mimic the same rules structure, as the prune part is in the operators() batch, which gets executed multiple times

Adding to this, I think a test like this would fail now (as in. limit won't be pruned), but work if it's separated:

FROM index | EVAL x = 5 | LIMIT 1 BY x

x should be propagated to LIMIT BY, and then pruned

I've taken the following steps:

Split the rules for this into PruneLiteralsInLimitBy and ReplaceLimitByExpressionWithEval. I also had to modify PropagateEvalFoldables to cascade the foldable constants into the groupings of Limit.

Added a bunch of tests in ReplaceLimitByExpressionWithEvalTests and PruneLiteralsInLimitByTests.

Took the opportunity to also reorganize the logical planner tests a bit better and moved some tests in PushDownAndCombineLimits.

ivancea · 2026-03-10T12:15:40Z

x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/plan/logical/Limit.java

+        this.groupings = groupings;
+    }
+
+    public Limit(Source source, Expression limit, LogicalPlan child, boolean duplicated, boolean local) {


We should consider removing this constructor now. It may uncover more changes, and keeping it could lead to bugs if we forget to pass the groupings

ivancea · 2026-03-10T12:27:02Z

x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/plan/physical/LimitExec.java

        this.estimatedRowSize = estimatedRowSize;
    }

+    public LimitExec(Source source, PhysicalPlan child, Expression limit, Integer estimatedRowSize) {


Same here, I would remove this one

...ck/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/planner/LocalExecutionPlanner.java

ivancea · 2026-03-10T12:51:54Z

x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/analysis/Verifier.java

        }
    }

+    private static void checkLimitBy(LogicalPlan plan, Failures failures) {


Note for reviewers: This is temporary until the TopN part is merged

ivancea · 2026-03-10T12:56:03Z

...in/esql/src/test/java/org/elasticsearch/xpack/esql/plan/logical/LimitSerializationTests.java

    }
+
+    public void testSerializationWithGroupingsOnOldVersion() {
+        TransportVersion oldVersion = TransportVersionUtils.getPreviousVersion(Limit.ESQL_LIMIT_BY);


Let's use randomTransportVersionBefore/randomTransportVersionBetween to test against multiple versions instead of a single fixed one.
Same for the other test

I've used this instead oldVersion = TransportVersionUtils.getPreviousVersion(Limit.ESQL_LIMIT_BY);

I also had to pick a fixed child for the Limit that serializes across all possible versions. I've gone with a EsRelation and EsSourceExec. Otherwise I was getting serialization errors for qualified attributes:

Trying to serialize an Attribute with a qualifier to an old node

ivancea · 2026-03-10T12:59:39Z

x-pack/plugin/esql/build.gradle

    '-package', 'org.elasticsearch.xpack.esql.parser',
    '-listener',
    '-visitor',
-    '-lib', outputPath,


What's this change?

Sorry I forgot to explain it.

antlr4 only accepts a lib argument, so here passing the second one overrides the first:

https://github.com/antlr/antlr4/blob/faa457ae5b9c39b572334814f0b36c855bc23010/tool/src/org/antlr/v4/Tool.java#L99

We should not have this in the build if it's doing nothing

ivancea

Review completed

...sql/src/main/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/CombineLimitTopN.java

ivancea · 2026-03-11T14:08:28Z

...c/main/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/HoistRemoteEnrichLimit.java

@@ -67,7 +68,7 @@ protected LogicalPlan rule(Enrich en, LogicalOptimizerContext ctx) {
            // Shouldn't actually throw because we checked seenLimits is not empty
            Limit lowestLimit = seenLimits.stream().min(Comparator.comparing(l -> (int) l.limit().fold(ctx.foldCtx()))).orElseThrow();
            // Insert new lowest limit on top of the Enrich, and mark it as duplicated since we don't want it to be pushed down
-            return new Limit(lowestLimit.source(), lowestLimit.limit(), transformLimits, true, false);
+            return new Limit(lowestLimit.source(), lowestLimit.limit(), transformLimits, List.of(), true, false);


This rule checks limits, but it's currently not checking for groups. The fact that it's checking for the lowest limit tells me this must be updated

I've added the check now. I think we need to add more tests with ENRICH that have LIMITs and LIMIT BYs

ivancea · 2026-03-11T14:15:16Z

...ck/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/planner/LocalExecutionPlanner.java

    private PhysicalOperation planLimit(LimitExec limit, LocalExecutionPlannerContext context) {
        PhysicalOperation source = plan(limit.child(), context);
-        return source.with(new LimitOperator.Factory((Integer) limit.limit().fold(context.foldCtx)), source.layout);
+        int limitValue = (Integer) limit.limit().fold(context.foldCtx);


For context, some time ago, we """decided""" to avoid folding and instead do something like (int) ((Literal)limit.limit()).value(). Not here especifically, but in general, so only the planner folds things, ideally.
That said, we didn't migrate most of the code yet, and I wouldn't do it here, as it's indeed existing code. Just in case

...t/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/PruneLiteralsInLimitByTests.java

...a/org/elasticsearch/xpack/esql/optimizer/rules/logical/ReplaceLimitByExpressionWithEval.java

x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/parser/StatementParserTests.java

coderabbitai

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/PushDownAndCombineLimits.java (1)

60-72: ⚠️ Potential issue | 🔴 Critical

Grouped LIMIT rewrites still assume plain LIMIT semantics.

The new grouping-attribute guard only protects the Eval/Project path. ENRICH can still push or duplicate LIMIT n BY ... below enrich-only fields, and the fork helper still injects plain new Limit(..., limit.limit(), forkBranch) copies. That can either leave the grouping unresolved or drop entire groups before the top grouped limit runs.

Safe guardrail

 } else if (unary instanceof Enrich enrich) {
+    if (groupingsReferenceAttributeDefinedByChild(limit, enrich)) {
+        return limit;
+    }
     if (enrich.mode() == Enrich.Mode.REMOTE) {
         return duplicateLimitAsFirstGrandchild(limit, true);
     } else {
         return enrich.replaceChild(limit.replaceChild(enrich.child()));
     }
 }
 ...
 private static LogicalPlan maybePushDownLimitToFork(Limit limit, Fork fork, LogicalOptimizerContext ctx) {
+    if (limit.groupings().isEmpty() == false) {
+        return limit;
+    }
     // TODO: there's no reason why UnionAll should not benefit from this optimization

Also applies to: 81-87, 118-123, 141-153

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In
`@x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/PushDownAndCombineLimits.java`
around lines 60 - 72, The grouped-LIMIT rewrite currently only protects the
Eval/Project path; extend the guard so nodes that can introduce/define
attributes (notably Enrich) are treated the same as
Eval/Project/RegexExtract/CompoundOutputEval/InferencePlan in
PushDownAndCombineLimits: update the conditional that checks "unary instanceof
..." to include Enrich and any other plan classes that define output attributes,
and ensure the evalAliasNeedsData and groupingsReferenceAttributeDefinedByChild
checks still apply. Also update the fork helper that creates new Limit(...)
copies so it carries over the grouping metadata (use limit.groupings() rather
than only limit.limit()) when duplicating/pushing limits, and ensure
combineLimits(...) logic preserves grouping semantics when merging limits. Refer
to PushDownAndCombineLimits, combineLimits, evalAliasNeedsData,
groupingsReferenceAttributeDefinedByChild and the fork helper that constructs
new Limit instances.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In
`@x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/PushDownAndCombineLimits.java`:
- Around line 60-72: The grouped-LIMIT rewrite currently only protects the
Eval/Project path; extend the guard so nodes that can introduce/define
attributes (notably Enrich) are treated the same as
Eval/Project/RegexExtract/CompoundOutputEval/InferencePlan in
PushDownAndCombineLimits: update the conditional that checks "unary instanceof
..." to include Enrich and any other plan classes that define output attributes,
and ensure the evalAliasNeedsData and groupingsReferenceAttributeDefinedByChild
checks still apply. Also update the fork helper that creates new Limit(...)
copies so it carries over the grouping metadata (use limit.groupings() rather
than only limit.limit()) when duplicating/pushing limits, and ensure
combineLimits(...) logic preserves grouping semantics when merging limits. Refer
to PushDownAndCombineLimits, combineLimits, evalAliasNeedsData,
groupingsReferenceAttributeDefinedByChild and the fork helper that constructs
new Limit instances.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 55946362-8f61-48ee-8860-c60cf8069c2b

📥 Commits

Reviewing files that changed from the base of the PR and between 0312fe4 and 3375850.

⛔ Files ignored due to path filters (2)

server/src/main/resources/transport/definitions/referable/esql_limit_by.csv is excluded by !**/*.csv
server/src/main/resources/transport/upper_bounds/9.4.csv is excluded by !**/*.csv

📒 Files selected for processing (92)

x-pack/plugin/esql/build.gradle
x-pack/plugin/esql/qa/testFixtures/src/main/java/org/elasticsearch/xpack/esql/EsqlTestUtils.java
x-pack/plugin/esql/qa/testFixtures/src/main/resources/limit.csv-spec
x-pack/plugin/esql/src/main/antlr/EsqlBaseParser.g4
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/EsqlCapabilities.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/analysis/Verifier.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/core/expression/Expressions.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/LogicalPlanOptimizer.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/CombineLimitTopN.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/HoistRemoteEnrichLimit.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/PropagateEvalFoldables.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/PruneColumns.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/PruneLiteralsInLimitBy.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/PushDownAndCombineLimits.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/ReplaceLimitByExpressionWithEval.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/PushLimitToSource.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/parser/EsqlBaseParser.interp
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/parser/EsqlBaseParser.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/parser/EsqlBaseParserBaseListener.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/parser/EsqlBaseParserBaseVisitor.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/parser/EsqlBaseParserListener.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/parser/EsqlBaseParserVisitor.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/parser/LogicalPlanBuilder.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/plan/logical/Limit.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/plan/physical/LimitExec.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/planner/LocalExecutionPlanner.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/planner/mapper/LocalMapper.java
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/planner/mapper/Mapper.java
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/analysis/AnalyzerTests.java
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/analysis/VerifierTests.java
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/datasources/SplitDiscoveryPhaseTests.java
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/LocalPhysicalPlanOptimizerTests.java
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/LogicalPlanOptimizerTests.java
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/PhysicalPlanOptimizerTests.java
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/PruneLiteralsInLimitByTests.java
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/PushDownAndCombineLimitsTests.java
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/ReplaceLimitByExpressionWithEvalTests.java
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/PushLimitToExternalSourceTests.java
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/parser/StatementParserTests.java
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/plan/logical/LimitSerializationTests.java
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/plan/physical/ExchangeSinkExecSerializationTests.java
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/plan/physical/LimitExecSerializationTests.java
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/plugin/AdaptiveStrategyTests.java
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/plugin/CoordinatorOnlyStrategyTests.java
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/plugin/ExternalDistributionTests.java
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/plugin/ExternalSourceDataNodeTests.java
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/golden_tests/PushdownGoldenTests/testFilterConjunctionPushableAndNonPushable/load/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/golden_tests/PushdownGoldenTests/testFilterDisjunctionPushableAndNonPushable/load/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/golden_tests/PushdownGoldenTests/testFilterDisjunctionPushableAndNonPushable/nullify/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/golden_tests/PushdownGoldenTests/testFilterNoPushdownWithUnmapped/load/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/golden_tests/PushdownGoldenTests/testFilterPushdownNoUnmapped/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/golden_tests/PushdownGoldenTests/testFilterPushdownNoUnmappedFilterOnly/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketNotTransformToQueryAndTagsWithFork/bucket/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketNotTransformToQueryAndTagsWithFork/date_trunc/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketNotTransformToQueryAndTagsWithFork/round_to/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketNotTransformToQueryAndTagsWithLookupJoin/bucket/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketNotTransformToQueryAndTagsWithLookupJoin/date_trunc/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketNotTransformToQueryAndTagsWithLookupJoin/round_to/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTags/bucket/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTags/date_trunc/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTags/round_to/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTagsWithEsFilter/bucket/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTagsWithEsFilter/date_trunc/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTagsWithEsFilter/round_to/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTagsWithMultipleAggregates/bucket/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTagsWithMultipleAggregates/date_trunc/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTagsWithMultipleAggregates/round_to/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTagsWithOtherPushdownFunctions/bucket/date_range/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTagsWithOtherPushdownFunctions/bucket/keyword_equality/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTagsWithOtherPushdownFunctions/bucket/keyword_match/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTagsWithOtherPushdownFunctions/date_trunc/date_range/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTagsWithOtherPushdownFunctions/date_trunc/keyword_equality/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTagsWithOtherPushdownFunctions/date_trunc/keyword_match/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTagsWithOtherPushdownFunctions/round_to/date_range/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTagsWithOtherPushdownFunctions/round_to/keyword_equality/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTagsWithOtherPushdownFunctions/round_to/keyword_match/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTagsWithWhereInsideAggregation/bucket/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTagsWithWhereInsideAggregation/date_trunc/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testDateTruncBucketTransformToQueryAndTagsWithWhereInsideAggregation/round_to/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testForkWithStatsCountStarDateTrunc/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testRoundToTransformToQueryAndTags/byte/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testRoundToTransformToQueryAndTags/date/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testRoundToTransformToQueryAndTags/date_nanos/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testRoundToTransformToQueryAndTags/double/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testRoundToTransformToQueryAndTags/float/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testRoundToTransformToQueryAndTags/half_float/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testRoundToTransformToQueryAndTags/integer/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testRoundToTransformToQueryAndTags/long/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testRoundToTransformToQueryAndTags/scaled_float/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testRoundToTransformToQueryAndTags/short/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/golden_tests/SubstituteRoundToGoldenTests/testSubqueryWithCountStarAndDateTrunc/local_physical_optimization.expected
x-pack/plugin/esql/src/test/resources/org/elasticsearch/xpack/esql/plugin/golden_tests/LateMaterializationPlannerGoldenTests/testTopNThenStats/physical_optimization.expected

💤 Files with no reviewable changes (1)

x-pack/plugin/esql/build.gradle

ivancea

Nice!

...ck/plugin/esql/qa/testFixtures/src/main/java/org/elasticsearch/xpack/esql/EsqlTestUtils.java

...t/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/PruneLiteralsInLimitByTests.java

ncordon · 2026-03-12T13:01:59Z

We are going to close this in favour of #144069

We've decided to split the LIMIT BY LimitBy and LimitByExec into separate planner nodes becuase the .groupings() check was polluting all of the code for Limit. And we do have lots of rules that manipulate Limit at the moment, so that felt safer and cleaner.

elasticsearchmachine added the v9.4.0 label Mar 6, 2026

ncordon force-pushed the esql-limit-by-planner branch 2 times, most recently from 3164b89 to 54e2846 Compare March 9, 2026 15:09

[ESQL] Adds parsing and planner wiring for LIMIT BY

9c3ef06

ncordon force-pushed the esql-limit-by-planner branch from 54e2846 to 9c3ef06 Compare March 9, 2026 15:49

ncordon added 3 commits March 9, 2026 16:55

Merge remote-tracking branch 'upstream/main' into esql-limit-by-planner

869254f

Small nits

7be9e5d

Adds more tests

08c9d83

ncordon force-pushed the esql-limit-by-planner branch from c182b39 to 08c9d83 Compare March 9, 2026 18:38

ncordon added >feature :Analytics/ES|QL AKA ESQL Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) labels Mar 9, 2026

Merge branch 'main' into esql-limit-by-planner

e8db931

ncordon requested review from Copilot and ivancea March 9, 2026 18:39

Copilot started reviewing on behalf of ncordon March 9, 2026 18:40 View session

Copilot AI reviewed Mar 9, 2026

View reviewed changes

ncordon and others added 4 commits March 10, 2026 00:49

Fixes tests

6665e6f

Fixes test

975cb89

Fixes capability description

3bbe10d

Merge branch 'main' into esql-limit-by-planner

dc80e02

ncordon marked this pull request as ready for review March 10, 2026 09:11

ncordon and others added 4 commits March 10, 2026 10:56

Adds tests for negative and zero limits

650d699

Adds tests for negative and zero limits

6cee286

Merge branch 'main' into esql-limit-by-planner

b96fda3

Merge branch 'main' into esql-limit-by-planner

376046f

julian-elastic reviewed Mar 10, 2026

View reviewed changes

x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/plan/logical/Limit.java Outdated Show resolved Hide resolved

julian-elastic approved these changes Mar 10, 2026

View reviewed changes

ivancea reviewed Mar 10, 2026

View reviewed changes

ncordon and others added 11 commits March 10, 2026 15:36

Addresses pr review feedback

8de248c

[CI] Auto commit changes from spotless

3a3663e

Fixes missing rule for LIMIT BY

9de8437

Merge remote-tracking branch 'upstream/main' into esql-limit-by-planner

118fce4

Moves tests to limit.csv-spec file

31821f3

Split planner rule into two

a07bed3

Removes Limit and LimitExec constructors without groupings

7955598

Adds test for PruneLiteralsInLimitBy and previous Eval

2252f9d

Reorganizes tests

614cf8e

Adds more tests

066ebb7

Tests random serialization version instead

e47b07d

ncordon force-pushed the esql-limit-by-planner branch from 1250225 to e47b07d Compare March 11, 2026 12:35

ncordon added 2 commits March 11, 2026 13:56

Merge remote-tracking branch 'upstream/main' into esql-limit-by-planner

a89c8fd

Changes outdated comment

26a0c76

ncordon requested review from ivancea and julian-elastic March 11, 2026 13:00

ivancea reviewed Mar 11, 2026

View reviewed changes

ncordon added 2 commits March 11, 2026 17:28

Addresses more pr review feedback

6851b6b

Merge remote-tracking branch 'upstream/main' into esql-limit-by-planner

3375850

coderabbitai bot reviewed Mar 11, 2026

View reviewed changes

ivancea approved these changes Mar 11, 2026

View reviewed changes

...ck/plugin/esql/qa/testFixtures/src/main/java/org/elasticsearch/xpack/esql/EsqlTestUtils.java Show resolved Hide resolved

...t/java/org/elasticsearch/xpack/esql/optimizer/rules/logical/PruneLiteralsInLimitByTests.java Show resolved Hide resolved

ncordon added 3 commits March 11, 2026 20:04

Fixes flaky test, adds one more assert to eval field checks

3bf8e8a

Gates ENRICH and FORK against simplifying LIMIT BY

609c51f

Merge remote-tracking branch 'upstream/main' into esql-limit-by-planner

28989be

ivancea approved these changes Mar 12, 2026

View reviewed changes

ncordon closed this Mar 12, 2026

		new ReplaceOrderByExpressionWithEval(),
		new ReplaceLimitByExpressionWithEval(),

Conversation

ncordon commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai bot commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 9, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elasticsearchmachine commented Mar 10, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

julian-elastic left a comment

Choose a reason for hiding this comment

Uh oh!

ivancea left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ncordon Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ivancea left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ncordon Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

ncordon commented Mar 6, 2026 •

edited

Loading

coderabbitai bot commented Mar 6, 2026 •

edited

Loading

ncordon Mar 10, 2026 •

edited

Loading

ncordon Mar 11, 2026 •

edited

Loading