Skip to content

[ESQL] Adds HeapAttack tests for LIMIT BY and SORT | LIMIT BY#145115

Merged
ncordon merged 23 commits intoelastic:mainfrom
ncordon:limit-by-heap-attack
Mar 31, 2026
Merged

[ESQL] Adds HeapAttack tests for LIMIT BY and SORT | LIMIT BY#145115
ncordon merged 23 commits intoelastic:mainfrom
ncordon:limit-by-heap-attack

Conversation

@ncordon
Copy link
Copy Markdown
Member

@ncordon ncordon commented Mar 27, 2026

Adds HeapAttackLimitByIT, covering circuit-breaking inside GroupedLimitOperator and GroupedTopNOperator. Shared helpers are moved to HeapAttackTestCase.

Paths tested:

  • GroupedLimitOperator: seenKeys hash table (wide group keys), GroupKeyEncoder scratch buffer (many large strings), large group limit (N in LIMIT N BY)
  • GroupedTopNOperator: TopNRow storage (many sort columns), keysHash (wide group keys), TopNQueue pre-allocation (many groups × large per-group limit) and GroupKeyEncoder scratch buffer (many large strings), large group limit (N in LIMIT N BY)

Path not tested: counts array in GroupedLimitOperator: only 4 bytes/group, so seenKeys always trips first.

Also adds assertCircuitBreaksVia(TryCircuitBreaking, String... classNames) which validates the exception stack trace contains the expected classes, and retries if a different part of the pipeline trips first.

Addresses https://github.com/elastic/esql-planning/issues/404

@ncordon ncordon added >non-issue Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) :Analytics/ES|QL AKA ESQL v9.4.0 labels Mar 27, 2026
@ncordon ncordon force-pushed the limit-by-heap-attack branch from 418e5eb to a25e1f5 Compare March 30, 2026 14:32
@ncordon ncordon marked this pull request as ready for review March 30, 2026 14:33
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@ncordon ncordon force-pushed the limit-by-heap-attack branch from ab709b2 to 91a4b1d Compare March 30, 2026 14:52
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds new ESQL heap-attack REST tests focused on circuit-breaking behavior in LIMIT BY and SORT | LIMIT BY operators, and factors shared heap-attack helpers into HeapAttackTestCase to support these tests.

Changes:

  • Add HeapAttackLimitByIT covering multiple circuit-breaking paths in GroupedLimitOperator and GroupedTopNOperator.
  • Add assertCircuitBreaksVia(...) plus shared dataset/query helpers to HeapAttackTestCase (and remove duplicates from HeapAttackIT).
  • Adjust compute/operator code to support the new tests (resource cleanup + class visibility) and add test module dependencies.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
x-pack/plugin/esql/compute/src/main/java/org/elasticsearch/compute/operator/topn/TopNRow.java Makes TopNRow public so heap-attack tests can reference it for stack-trace assertions.
x-pack/plugin/esql/compute/src/main/java/org/elasticsearch/compute/operator/topn/TopNQueue.java Makes TopNQueue public so heap-attack tests can reference it for stack-trace assertions.
x-pack/plugin/esql/compute/src/main/java/org/elasticsearch/compute/operator/GroupedLimitOperator.java Ensures counts is released on constructor failure.
test/external-modules/esql-heap-attack/src/javaRestTest/java/org/elasticsearch/xpack/esql/heap_attack/HeapAttackTestCase.java Adds assertCircuitBreaksVia(...) and shared index/query helpers (initManyLongs, makeManyLongs, etc.).
test/external-modules/esql-heap-attack/src/javaRestTest/java/org/elasticsearch/xpack/esql/heap_attack/HeapAttackLimitByIT.java New integration tests for heap-attack coverage of LIMIT BY and SORT | LIMIT BY.
test/external-modules/esql-heap-attack/src/javaRestTest/java/org/elasticsearch/xpack/esql/heap_attack/HeapAttackIT.java Removes duplicated helper methods now hosted in HeapAttackTestCase.
test/external-modules/esql-heap-attack/build.gradle Adds test dependencies on :x-pack:plugin:esql:compute and :libs:swisshash.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Copy Markdown
Contributor

@julian-elastic julian-elastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good, should be good to merge after the comments are addressed!

@ncordon ncordon merged commit ddb3fe6 into elastic:main Mar 31, 2026
35 checks passed
ncordon added a commit to ncordon/elasticsearch that referenced this pull request Apr 1, 2026
mromaios pushed a commit to mromaios/elasticsearch that referenced this pull request Apr 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/ES|QL AKA ESQL >non-issue Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v9.4.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants