Skip to content

Conversation

@ahkcs
Copy link
Contributor

@ahkcs ahkcs commented Aug 28, 2025

Description

Adding big5 queries used for performance testing in the new features implemented for the bin command.

Related Issues

#3878

Signed-off-by: Kai Huang <[email protected]>
Signed-off-by: Kai Huang <[email protected]>
@LantaoJin
Copy link
Member

please update CalcitePPLBig5IT

@LantaoJin LantaoJin added the testing Related to improving software testing label Aug 29, 2025
Signed-off-by: Kai Huang <[email protected]>

@Test
public void bin_aligntime() throws IOException {
String ppl = sanitize(loadFromFile("big5/queries/bin_aligntime.ppl"));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QQ: Why do we need to load from separate file? Query seems simple and it should be simpler to write here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we are adding this IT to verify if the queries we put under the big5 queries folder works, also to be consistent with merged PR: #4127

Copy link
Collaborator

@Swiddis Swiddis Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^ keeping it separate helps us copy the queries as benchmark cases, we want to ideally just copy the contents of big5/queries and generate workloads from it.

Swiddis
Swiddis previously approved these changes Sep 3, 2025
@Swiddis Swiddis enabled auto-merge (squash) September 3, 2025 20:54
Copy link
Collaborator

@noCharger noCharger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ahkcs What queries generate the most overhead? If the implementation does not diverge, they can unite into one.

auto-merge was automatically disabled September 4, 2025 18:37

Head branch was pushed to by a user without write access

@ahkcs
Copy link
Contributor Author

ahkcs commented Sep 4, 2025

@ahkcs What queries generate the most overhead? If the implementation does not diverge, they can unite into one.

I have removed the other test cases and kept

  1. bin_span_time.ppl - Time-based binning
  2. bin_span_log.ppl - Logarithmic binning
  3. bin_bins.ppl - Magnitude-based binning

Signed-off-by: Kai Huang <[email protected]>
@Swiddis Swiddis enabled auto-merge (squash) September 5, 2025 17:35
@Swiddis Swiddis merged commit 945235a into opensearch-project:main Sep 5, 2025
23 checks passed
joshuali925 pushed a commit that referenced this pull request Sep 16, 2025
* Bin command big5 queries

Signed-off-by: Kai Huang <[email protected]>

* update IT

Signed-off-by: Kai Huang <[email protected]>

* fix

Signed-off-by: Kai Huang <[email protected]>

* remove tests

Signed-off-by: Kai Huang <[email protected]>

---------

Signed-off-by: Kai Huang <[email protected]>
Signed-off-by: Kai Huang <[email protected]>
joshuali925 pushed a commit that referenced this pull request Sep 24, 2025
* Doc enhancement for eventstats and bin command (#4117)

* distinct_count doc for eventstats

Signed-off-by: Kai Huang <[email protected]>

* doc enhancement

Signed-off-by: Kai Huang <[email protected]>

* add fields for consistency between different Java versions

Signed-off-by: Kai Huang <[email protected]>

* remove changes

Signed-off-by: Kai Huang <[email protected]>

* add bin to index.rst

Signed-off-by: Kai Huang <[email protected]>

* add link

Signed-off-by: Kai Huang <[email protected]>

* fix

Signed-off-by: Kai Huang <[email protected]>

---------

Signed-off-by: Kai Huang <[email protected]>

* Implement `Append` command with Calcite (#4123)

* Implement Append Command

Signed-off-by: Songkan Tang <[email protected]>

* Fix spotless check

Signed-off-by: Songkan Tang <[email protected]>

* Rephrase append.rst

Signed-off-by: Songkan Tang <[email protected]>

* Support subsearch different index for append command

Signed-off-by: Songkan Tang <[email protected]>

* Fix some tests and add cross cluster IT

Signed-off-by: Songkan Tang <[email protected]>

* Not support empty subsearch input for now

Signed-off-by: Songkan Tang <[email protected]>

* Fix doctest

Signed-off-by: Songkan Tang <[email protected]>

* Support empty source edge case

Signed-off-by: Songkan Tang <[email protected]>

* Fix anonymizer tests

Signed-off-by: Songkan Tang <[email protected]>

* Add missing test cases for nested join or lookup command in appended subsearch

Signed-off-by: Songkan Tang <[email protected]>

* Fix compile issue

Signed-off-by: Songkan Tang <[email protected]>

---------

Signed-off-by: Songkan Tang <[email protected]>

* `Bin` command big5 queries (#4163)

* Bin command big5 queries

Signed-off-by: Kai Huang <[email protected]>

* update IT

Signed-off-by: Kai Huang <[email protected]>

* fix

Signed-off-by: Kai Huang <[email protected]>

* remove tests

Signed-off-by: Kai Huang <[email protected]>

---------

Signed-off-by: Kai Huang <[email protected]>
Signed-off-by: Kai Huang <[email protected]>

* Don't recreate indices on every test (#4222)

Signed-off-by: Simeon Widdis <[email protected]>

* Enable pushdown optimization for filtered aggregation (#4213)

* Enable filtered aggregation pushdown

Signed-off-by: Chen Dai <[email protected]>

* Add basic UT and ignore IT for now

Signed-off-by: Chen Dai <[email protected]>

* Enable aggregate case to filter rule and fix UT and IT

Signed-off-by: Chen Dai <[email protected]>

* Add expected json file for no pushdown test

Signed-off-by: Chen Dai <[email protected]>

* Remove unnecessary aggregate case to filter rule

Signed-off-by: Chen Dai <[email protected]>

* Add UT for IS_TRUE support

Signed-off-by: Chen Dai <[email protected]>

* Add more explain IT

Signed-off-by: Chen Dai <[email protected]>

* Refactor UT

Signed-off-by: Chen Dai <[email protected]>

* Extract aggregate filter analyzer abstraction

Signed-off-by: Chen Dai <[email protected]>

* Add more UT

Signed-off-by: Chen Dai <[email protected]>

* Refactor UT with fluent API

Signed-off-by: Chen Dai <[email protected]>

* Add UT for distinct count

Signed-off-by: Chen Dai <[email protected]>

* Address comment by adding UT for script filter pushdown

Signed-off-by: Chen Dai <[email protected]>

* Fix spotless

Signed-off-by: Chen Dai <[email protected]>

---------

Signed-off-by: Chen Dai <[email protected]>

* Split up our test actions into unit, integ, and doctest. (#4193)

* Run unit test suites in parallel

Signed-off-by: Simeon Widdis <[email protected]>

* Split out our test actions

Signed-off-by: Simeon Widdis <[email protected]>

* Make unit test step run in parallel

Signed-off-by: Simeon Widdis <[email protected]>

* Fix removed bwc tests

Signed-off-by: Simeon Widdis <[email protected]>

* Add another missing parallel flag

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>
Signed-off-by: Simeon Widdis <[email protected]>

* [Feature] Core Implementation of `rex` Command In PPL (#4109)

* rex - initial implementation

Signed-off-by: Jialiang Liang <[email protected]>

* stop using utils

Signed-off-by: Jialiang Liang <[email protected]>

* fix spotless check

Signed-off-by: Jialiang Liang <[email protected]>

* offset_field - initial implementation

Signed-off-by: Jialiang Liang <[email protected]>

* max_match - initial implementation

Signed-off-by: Jialiang Liang <[email protected]>

* sed - initial implementation

Signed-off-by: Jialiang Liang <[email protected]>

* fix name capture group for extraction

Signed-off-by: Jialiang Liang <[email protected]>

* add rex rst doc

Signed-off-by: Jialiang Liang <[email protected]>

* IT - initial setup

Signed-off-by: Jialiang Liang <[email protected]>

* add a analyzer test for legacy engine

Signed-off-by: Jialiang Liang <[email protected]>

* Add UT for rex

Signed-off-by: Jialiang Liang <[email protected]>

* sed - add pushdown for sed and explain IT and IT with fix

Signed-off-by: Jialiang Liang <[email protected]>

* anonymizer - add rex for anonymizer and test

Signed-off-by: Jialiang Liang <[email protected]>

* Add cross cluster IT for rex

Signed-off-by: Jialiang Liang <[email protected]>

* peng - resolve comments for rst doc 0

Signed-off-by: Jialiang Liang <[email protected]>

* peng - address some comments 1

Signed-off-by: Jialiang Liang <[email protected]>

* peng - resolve comment in rst doc to add a java doc link

Signed-off-by: Jialiang Liang <[email protected]>

* kai - modify the bin ast builder test

Signed-off-by: Jialiang Liang <[email protected]>

* peng - fix the extraction behavior without filter even when there is zero match

Signed-off-by: Jialiang Liang <[email protected]>

* fix rex explain no pushdown

Signed-off-by: Jialiang Liang <[email protected]>

* change the offset val output format

Signed-off-by: Jialiang Liang <[email protected]>

* fix rst file

Signed-off-by: Jialiang Liang <[email protected]>

* peng - SWITCH TO USE CALCITE NATIVE OPERATORS

Signed-off-by: Jialiang Liang <[email protected]>

* Peng - fix tests after operator change

Signed-off-by: Jialiang Liang <[email protected]>

* support mode=extract and update doc

Signed-off-by: Jialiang Liang <[email protected]>

* fix the issue after rebase

Signed-off-by: Jialiang Liang <[email protected]>

* peng - enforce specifying field in antlr for now

Signed-off-by: Jialiang Liang <[email protected]>

* relocate rex cmd IT

Signed-off-by: Jialiang Liang <[email protected]>

* peng - simplify vistFunciton

Signed-off-by: Jialiang Liang <[email protected]>

* peng - add UT for RexExtractMultiFunction

Signed-off-by: Jialiang Liang <[email protected]>

* peng - add UT RexOffsetFunction

Signed-off-by: Jialiang Liang <[email protected]>

* fix some tests

Signed-off-by: Jialiang Liang <[email protected]>

* DECOUPLE SED + OFFSET FIELD

Signed-off-by: Jialiang Liang <[email protected]>

* Improve error handling for extract

Signed-off-by: Jialiang Liang <[email protected]>

* add this rex rst into index

Signed-off-by: Jialiang Liang <[email protected]>

* fix return type in extract multi

Signed-off-by: Jialiang Liang <[email protected]>

* add rex doc into doc test

Signed-off-by: Jialiang Liang <[email protected]>

* fix doc test

Signed-off-by: Jialiang Liang <[email protected]>

* Fix linting

Signed-off-by: Jialiang Liang <[email protected]>

* fix rebase issue

Signed-off-by: Jialiang Liang <[email protected]>

* fix regex anonymizer tests

Signed-off-by: Jialiang Liang <[email protected]>

* fix analyzer test and setup to use util function

Signed-off-by: Jialiang Liang <[email protected]>

* lint fix

Signed-off-by: Jialiang Liang <[email protected]>

* fix doc test

Signed-off-by: Jialiang Liang <[email protected]>

* Add max match limit implementation

Signed-off-by: Jialiang Liang <[email protected]>

* fix anonymizer test

Signed-off-by: Jialiang Liang <[email protected]>

* peng - simplify if

Signed-off-by: Jialiang Liang <[email protected]>

* peng - make extract multi to only handle the case of max_match > 1

Signed-off-by: Jialiang Liang <[email protected]>

---------

Signed-off-by: Jialiang Liang <[email protected]>

* Add wildcard support for rename command (#4019)

* add wildcard support for rename

Signed-off-by: Ritvi Bhatt <[email protected]>

* fix calcite wildcard support and add tests

Signed-off-by: Ritvi Bhatt <[email protected]>

* fix formatting

Signed-off-by: Ritvi Bhatt <[email protected]>

* add check to analyzer

Signed-off-by: Ritvi Bhatt <[email protected]>

* update doc formatting

Signed-off-by: Ritvi Bhatt <[email protected]>

* remove v2 engine wildcard support

Signed-off-by: Ritvi Bhatt <[email protected]>

* update doc

Signed-off-by: Ritvi Bhatt <[email protected]>

* fix formatting

Signed-off-by: Ritvi Bhatt <[email protected]>

* support cascading rename

Signed-off-by: Ritvi Bhatt <[email protected]>

* update formatting

Signed-off-by: Ritvi Bhatt <[email protected]>

* add cross cluster test

Signed-off-by: Ritvi Bhatt <[email protected]>

* add test for cascading rename

Signed-off-by: Ritvi Bhatt <[email protected]>

* fix formatting

Signed-off-by: Ritvi Bhatt <[email protected]>

* add test for cascading rename

Signed-off-by: Ritvi Bhatt <[email protected]>

* change behavior for renaming existing fields

Signed-off-by: Ritvi Bhatt <[email protected]>

* add tests and update docs

Signed-off-by: Ritvi Bhatt <[email protected]>

* update docs

Signed-off-by: Ritvi Bhatt <[email protected]>

* update docs

Signed-off-by: Ritvi Bhatt <[email protected]>

* fix renaming to same name

Signed-off-by: Ritvi Bhatt <[email protected]>

* fix behavior for consecutive wildcards/address comments

Signed-off-by: Ritvi Bhatt <[email protected]>

* add back import

Signed-off-by: Ritvi Bhatt <[email protected]>

* fix doc

Signed-off-by: Ritvi Bhatt <[email protected]>

* fix doc

Signed-off-by: Ritvi Bhatt <[email protected]>

* fix formatting

Signed-off-by: Ritvi Bhatt <[email protected]>

---------

Signed-off-by: Ritvi Bhatt <[email protected]>
Signed-off-by: ritvibhatt <[email protected]>

* Add support for `median(<value>)` (#4234)

* First revision

Signed-off-by: Aaron Alvarez <[email protected]>

* Fixing documentation

Signed-off-by: Aaron Alvarez <[email protected]>

* Removing unnecessary comments

Signed-off-by: Aaron Alvarez <[email protected]>

* Fixinf stats.rst documentation

Signed-off-by: Aaron Alvarez <[email protected]>

* Fixing documentation

Signed-off-by: Aaron Alvarez <[email protected]>

* Addressing comments

Signed-off-by: Aaron Alvarez <[email protected]>

---------

Signed-off-by: Aaron Alvarez <[email protected]>
Signed-off-by: Aaron Alvarez <[email protected]>
Co-authored-by: Aaron Alvarez <[email protected]>

* Dynamic source selector (#4116)

Signed-off-by: Vamsi Manohar <[email protected]>

* Add gitignore (#4258)

Signed-off-by: Tomoyuki Morita <[email protected]>

* Support join field list and join options (#3803)

* Support join field list and join options

Signed-off-by: Lantao Jin <[email protected]>

* Add SPL-compatible syntax setting

Signed-off-by: Lantao Jin <[email protected]>

* Revert SPL settings

Signed-off-by: Lantao Jin <[email protected]>

* Fix IT

Signed-off-by: Lantao Jin <[email protected]>

* Fix IT

Signed-off-by: Lantao Jin <[email protected]>

* Support max=n option

Signed-off-by: Lantao Jin <[email protected]>

* support max=n in sql-like join syntax

Signed-off-by: Lantao Jin <[email protected]>

* Add Explain IT for new join syntax

Signed-off-by: Lantao Jin <[email protected]>

* Refactor the user doc

Signed-off-by: Lantao Jin <[email protected]>

* Fix conflicts

Signed-off-by: Lantao Jin <[email protected]>

* Fix conflicts

Signed-off-by: Lantao Jin <[email protected]>

* Disable the collapse pushdown

Signed-off-by: Lantao Jin <[email protected]>

* refactor

Signed-off-by: Lantao Jin <[email protected]>

* Fix IT

Signed-off-by: Lantao Jin <[email protected]>

---------

Signed-off-by: Lantao Jin <[email protected]>

* Support first/last aggregate functions for PPL (#4223)

* Support first/last aggregation functions for PPL

Signed-off-by: Kai Huang <[email protected]>

* Support null

Signed-off-by: Kai Huang <[email protected]>

* remove legacy

Signed-off-by: Kai Huang <[email protected]>

* update doc

Signed-off-by: Kai Huang <[email protected]>

* fix doctest

Signed-off-by: Kai Huang <[email protected]>

* fix stats.rst file

Signed-off-by: Kai Huang <[email protected]>

* fixes

Signed-off-by: Kai Huang <[email protected]>

* move pushdown logic to AggregateAnalyzer

Signed-off-by: Kai Huang <[email protected]>

* fix IT and update null handling

Signed-off-by: Kai Huang <[email protected]>

* add test cases for null handling

Signed-off-by: Kai Huang <[email protected]>

* handle parallelism

Signed-off-by: Kai Huang <[email protected]>

* Simplify CalciteExplainIT and add UT for AggregateAnalyzer

Signed-off-by: Kai Huang <[email protected]>

# Conflicts:
#	opensearch/src/test/java/org/opensearch/sql/opensearch/request/AggregateAnalyzerTest.java

* fixes

Signed-off-by: Kai Huang <[email protected]>

---------

Signed-off-by: Kai Huang <[email protected]>

* Fix gitignore to ignore symbolic link (#4263)

add comment

Signed-off-by: Tomoyuki Morita <[email protected]>

* Push down limit operator into aggregation bucket size (#4228)

* Push down limit operator into aggregation bucket size

Signed-off-by: Heng Qian <[email protected]>

* Fix IT

Signed-off-by: Heng Qian <[email protected]>

* Fix robust issue in OpenSearchLimitIndexScanRule

Signed-off-by: Heng Qian <[email protected]>

* Refine comments

Signed-off-by: Heng Qian <[email protected]>

---------

Signed-off-by: Heng Qian <[email protected]>

* Fix the IT issue caused by merging conflict (#4270)

Signed-off-by: Heng Qian <[email protected]>

* Print links to test logs after integTest (#4273)

* Print links to test logs after integTest

Signed-off-by: Tomoyuki Morita <[email protected]>

* print even when tets failed

Signed-off-by: Tomoyuki Morita <[email protected]>

---------

Signed-off-by: Tomoyuki Morita <[email protected]>

* [Feature] Implementation of mode `sed` and `offset_field` in rex PPL command (#4241)

* [Feature] Implementation of mode sed and offset_field in rex PPL command

Signed-off-by: Jialiang Liang <[email protected]>

* update rex rst doc

Signed-off-by: Jialiang Liang <[email protected]>

* chen - address comment and merge grammar in parser

Signed-off-by: Jialiang Liang <[email protected]>

* chen - limit offset field only in extraction mode

Signed-off-by: Jialiang Liang <[email protected]>

* chen - specify exception type of o_f UDF

Signed-off-by: Jialiang Liang <[email protected]>

* chen - add exception type of o_f UDF - 2

Signed-off-by: Jialiang Liang <[email protected]>

* chen - add exception type of o_f UDF - also fix the test

Signed-off-by: Jialiang Liang <[email protected]>

* chen - alphabetical order of o_f return

Signed-off-by: Jialiang Liang <[email protected]>

---------

Signed-off-by: Jialiang Liang <[email protected]>

* Add earliest/latest aggregate function for eventstats PPL command (#4212)

* Add earliest/latest aggregate function for eventstats command

Signed-off-by: Tomoyuki Morita <[email protected]>

* update docs

Signed-off-by: Tomoyuki Morita <[email protected]>

* Minor refactoring

Signed-off-by: Tomoyuki Morita <[email protected]>

* Fix doctest

Signed-off-by: Tomoyuki Morita <[email protected]>

* Simplify logics

Signed-off-by: Tomoyuki Morita <[email protected]>

* Revert visitWindowFunction

Signed-off-by: Tomoyuki Morita <[email protected]>

* Add sort to some examples

Signed-off-by: Tomoyuki Morita <[email protected]>

* Refactor tests

Signed-off-by: Tomoyuki Morita <[email protected]>

* Fix argument validation error (WIP)

Signed-off-by: Tomoyuki Morita <[email protected]>

* Add argument validation for window functions

Signed-off-by: Tomoyuki Morita <[email protected]>

* Fix validation

Signed-off-by: Tomoyuki Morita <[email protected]>

* Fix tests

Signed-off-by: Tomoyuki Morita <[email protected]>

* Fix tests and refactor

Signed-off-by: Tomoyuki Morita <[email protected]>

* Fix test

Signed-off-by: Tomoyuki Morita <[email protected]>

* Fix merge issue

Signed-off-by: Tomoyuki Morita <[email protected]>

---------

Signed-off-by: Tomoyuki Morita <[email protected]>

* Speed up aggregation pushdown for single group-by expression (#3550)

* Speed up aggregation pushdown for single group-by expression

Signed-off-by: Lantao Jin <[email protected]>

* Add configs nullable_bucket

Signed-off-by: Lantao Jin <[email protected]>

* Fix IT

Signed-off-by: Lantao Jin <[email protected]>

* revert typo

Signed-off-by: Lantao Jin <[email protected]>

* Fix conflicts error

Signed-off-by: Lantao Jin <[email protected]>

* fix unit tests

Signed-off-by: Lantao Jin <[email protected]>

* Fix order

Signed-off-by: Lantao Jin <[email protected]>

* Fix UT

Signed-off-by: Lantao Jin <[email protected]>

* Fix UT in windows

Signed-off-by: Lantao Jin <[email protected]>

* fix compile error of conflicts

Signed-off-by: Lantao Jin <[email protected]>

* Add more ITs after merging push down limit to agg buckets

Signed-off-by: Lantao Jin <[email protected]>

* fix IT

Signed-off-by: Lantao Jin <[email protected]>

* address comments

Signed-off-by: Lantao Jin <[email protected]>

* Clear sorts in source builder for aggregation pushdown

Signed-off-by: Lantao Jin <[email protected]>

* Delete the TODO of v2, it's resolved now

Signed-off-by: Lantao Jin <[email protected]>

* fix doctest

Signed-off-by: Lantao Jin <[email protected]>

---------

Signed-off-by: Lantao Jin <[email protected]>

* Introduce YAML formatter for better testing/debugging (#4274)

* Implement YamlFormatter

Signed-off-by: Tomoyuki Morita <[email protected]>

* Enable YAML based plan comparison in tests

Signed-off-by: Tomoyuki Morita <[email protected]>

* Fix line break issue in Windows

Signed-off-by: Tomoyuki Morita <[email protected]>

* Minor fix in test case

Signed-off-by: Tomoyuki Morita <[email protected]>

* Fix line break issue

Signed-off-by: Tomoyuki Morita <[email protected]>

* Fix comment

Signed-off-by: Tomoyuki Morita <[email protected]>

---------

Signed-off-by: Tomoyuki Morita <[email protected]>

* doctest: Use 1.0 branch instead of main (#4219)

Signed-off-by: Simeon Widdis <[email protected]>

* Fix doctest (#4292)

Signed-off-by: Simeon Widdis <[email protected]>

* Search Command Revamp (#4152)

Signed-off-by: Vamsi Manohar <[email protected]>

* `mvjoin` support in PPL Caclite (#4217)

* mvjoin support in PPL Caclite

Signed-off-by: ps48 <[email protected]>

* fix texts

Signed-off-by: ps48 <[email protected]>

* update docs

Signed-off-by: ps48 <[email protected]>

* update doc examples

Signed-off-by: ps48 <[email protected]>

* rebase main, update test

Signed-off-by: ps48 <[email protected]>

* update test with real array fields

Signed-off-by: ps48 <[email protected]>

* use verifyQueryThrowsException in CalcitePPLFunctionTypeTest

Signed-off-by: ps48 <[email protected]>

* spotless check fix

Signed-off-by: ps48 <[email protected]>

* remove string,string registration for mvjoin

Signed-off-by: ps48 <[email protected]>

* remove string,string test

Signed-off-by: ps48 <[email protected]>

---------

Signed-off-by: ps48 <[email protected]>

* strftime function implementation (#4106)

Signed-off-by: Vamsi Manohar <[email protected]>

* Add non-numeric field support for max/min functions (#4281)

* add non-numeric support for max/min

Signed-off-by: Ritvi Bhatt <[email protected]>

* fix mixed field behavior

Signed-off-by: Ritvi Bhatt <[email protected]>

* update doc

Signed-off-by: Ritvi Bhatt <[email protected]>

* update doc

Signed-off-by: Ritvi Bhatt <[email protected]>

* update formatting

Signed-off-by: Ritvi Bhatt <[email protected]>

* add tests

Signed-off-by: Ritvi Bhatt <[email protected]>

* fix formatting

Signed-off-by: Ritvi Bhatt <[email protected]>

* empty

Signed-off-by: Ritvi Bhatt <[email protected]>

* fix formatting

Signed-off-by: Ritvi Bhatt <[email protected]>

* fix

Signed-off-by: Ritvi Bhatt <[email protected]>

* support ip type max/min

Signed-off-by: Ritvi Bhatt <[email protected]>

* fix formatting

Signed-off-by: Ritvi Bhatt <[email protected]>

* use tophitsparser

Signed-off-by: Ritvi Bhatt <[email protected]>

* remove v2 explain

Signed-off-by: Ritvi Bhatt <[email protected]>

* check for numeric fields for native max/min

Signed-off-by: Ritvi Bhatt <[email protected]>

* change names

Signed-off-by: Ritvi Bhatt <[email protected]>

* fix type checking

Signed-off-by: Ritvi Bhatt <[email protected]>

---------

Signed-off-by: Ritvi Bhatt <[email protected]>
Signed-off-by: ritvibhatt <[email protected]>

* Add  `values` stats function with UDAF (#4276)

* Add  stats function

Signed-off-by: ps48 <[email protected]>

* add settings for max values

Signed-off-by: ps48 <[email protected]>

* update functiontypetest IT

Signed-off-by: ps48 <[email protected]>

* update documentation for values settings

Signed-off-by: ps48 <[email protected]>

* update the rst docs, remove settingsholder

Signed-off-by: ps48 <[email protected]>

* update AST additions

Signed-off-by: ps48 <[email protected]>

* updated the IT testValuesFunctionGroupBy

Signed-off-by: ps48 <[email protected]>

---------

Signed-off-by: ps48 <[email protected]>

* Support ISO8601-formatted string in PPL (#4246)

* Support parsing ISO 8601 datetime format for timestamp value

Signed-off-by: Yuanchun Shen <[email protected]>

* Modify tests for ISO 8601 timestamp input

Signed-off-by: Yuanchun Shen <[email protected]>

* Add support of iso 8601 date string to date and time

- add an IT for date time comparison with iso 8601 formatted literal

Signed-off-by: Yuanchun Shen <[email protected]>

---------

Signed-off-by: Yuanchun Shen <[email protected]>

* Push down project operator with non-identity projections into scan (#4279)

* Support project push down after aggregation

Signed-off-by: Heng Qian <[email protected]>

* Push down project operator with non-identity projections into scan

Signed-off-by: Heng Qian <[email protected]>

* Fix IT

Signed-off-by: Heng Qian <[email protected]>

* Also changing plan from merging main

Signed-off-by: Heng Qian <[email protected]>

* Fix IT

Signed-off-by: Heng Qian <[email protected]>

* Fix 4296

Signed-off-by: Heng Qian <[email protected]>

---------

Signed-off-by: Heng Qian <[email protected]>

* Add spotless precommit hook + license check (#4306)

* Add spotless precommit hook

Signed-off-by: Simeon Widdis <[email protected]>

* Decouple plugin spotless versions + upgrade spotless

Signed-off-by: Simeon Widdis <[email protected]>

* Enable license headers everywhere

Signed-off-by: Simeon Widdis <[email protected]>

* Remove a redundant comment

Signed-off-by: Simeon Widdis <[email protected]>

* Fix removed additional licenses

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>

* Add Ryan as a maintainer (#4257)

Signed-off-by: Simeon Widdis <[email protected]>

* Spotless precommit: apply instead of check (#4320)

* Add merge_group trigger to test workflows (#4216)

* Update grammar files and developer guide (#4301)

* Update grammar files and developer guide

Signed-off-by: Kai Huang <[email protected]>

* fix

Signed-off-by: Kai Huang <[email protected]>

---------

Signed-off-by: Kai Huang <[email protected]>

* Fix geopoiint issue in complex data types (#4325)

Signed-off-by: Vamsi Manohar <[email protected]>

* [Doc] Correct the comparision table for rex doc (#4321)

* [Doc] Correct the comparision table for rex doc

Signed-off-by: Jialiang Liang <[email protected]>

* peng - remove non support feature from comparison table

Signed-off-by: Jialiang Liang <[email protected]>

---------

Signed-off-by: Jialiang Liang <[email protected]>

* Add splunk to ppl cheat sheet (#3726)

* update with latest ppl commands and function improvement

Signed-off-by: Peng Huo <[email protected]>

* Address comments

Signed-off-by: Peng Huo <[email protected]>

---------

Signed-off-by: Peng Huo <[email protected]>

* Date/Time based Span aggregation should always not present null bucket (#4327)

* Updating coalesce documentation (#4305)

Co-authored-by: Aaron Alvarez <[email protected]>

* Support serializing & deserializing UDTs when pushing down scripts (#4245)

* Support serializing & deserializing UDTs

Signed-off-by: Yuanchun Shen <[email protected]>

* Update explain ITs

Signed-off-by: Yuanchun Shen <[email protected]>

* Push down UDT types as string types for comparison operators

Signed-off-by: Yuanchun Shen <[email protected]>

* Separate test cases and add an ignored IT

Signed-off-by: Yuanchun Shen <[email protected]>

* Correct the handling of UDT in CalciteScriptEngine by substituting calcite's type factory with OpenSearchTypeFactory

Signed-off-by: Yuanchun Shen <[email protected]>

* Fix deserialization for IP

Signed-off-by: Yuanchun Shen <[email protected]>

* Remove testExplainPushDownScriptsContainingUDT in v2

Signed-off-by: Yuanchun Shen <[email protected]>

* Enable testLimitAfterAggregation in CalcitePPLAggregationIT

Signed-off-by: Yuanchun Shen <[email protected]>

* Unit test serialize map and array types

Signed-off-by: Yuanchun Shen <[email protected]>

* Fix deeper level deserialization of UDTs

Signed-off-by: Yuanchun Shen <[email protected]>

* Add a yaml test for issue 4322

Signed-off-by: Yuanchun Shen <[email protected]>

* Add a test case for issue 4340

Signed-off-by: Yuanchun Shen <[email protected]>

* Remove redundant classes

Signed-off-by: Yuanchun Shen <[email protected]>

---------

Signed-off-by: Yuanchun Shen <[email protected]>

* change Anonymizer to mask PPL (#4352)

* change Anonymizer

Signed-off-by: xinyual <[email protected]>

* fix case

Signed-off-by: xinyual <[email protected]>

---------

Signed-off-by: xinyual <[email protected]>

* [Feature][Enhancement] Enhance patterns command with additional sample_logs output field (#4155)

* Enhance patterns command with additional sample_logs output field

Signed-off-by: Songkan Tang <[email protected]>

* Reorder agg fields for simple_pattern

Signed-off-by: Songkan Tang <[email protected]>

* Test fix after previous fix to not drop group by list

Signed-off-by: Songkan Tang <[email protected]>

---------

Signed-off-by: Songkan Tang <[email protected]>

* Optimize count aggregation performance by utilizing native doc_count in v3 (#4337)

* Optimize bucket aggregation performance by utilizing native doc_count in v3

Signed-off-by: Lantao Jin <[email protected]>

* fix UT

Signed-off-by: Lantao Jin <[email protected]>

* Fix issue of count(FIELD)

Signed-off-by: Lantao Jin <[email protected]>

* fix comments

Signed-off-by: Lantao Jin <[email protected]>

* Fix typo

Signed-off-by: Lantao Jin <[email protected]>

* revert the doc_count pushdown for count(FIELD) by EXPR

Signed-off-by: Lantao Jin <[email protected]>

* Support pushdown count aggregation in no bucket aggregation to hits.total.value

Signed-off-by: Lantao Jin <[email protected]>

---------

Signed-off-by: Lantao Jin <[email protected]>

* No index found with given index pattern should throw IndexNotFoundException (#4369)

* No index found with given index pattern should throw IndexNotFoundException

Signed-off-by: Lantao Jin <[email protected]>

* Add UT

Signed-off-by: Lantao Jin <[email protected]>

---------

Signed-off-by: Lantao Jin <[email protected]>

* Push down stats with bins on time field into auto_date_histogram (#4329)

* Push down stats with bins on time field into auto_date_histogram

Signed-off-by: Heng Qian <[email protected]>

* Prevent pushing down multiple group-by with bins in advance.

Signed-off-by: Heng Qian <[email protected]>

* Remove useless code

Signed-off-by: Heng Qian <[email protected]>

* Fix IT after merging main

Signed-off-by: Heng Qian <[email protected]>

---------

Signed-off-by: Heng Qian <[email protected]>

---------

Signed-off-by: Kai Huang <[email protected]>
Signed-off-by: Songkan Tang <[email protected]>
Signed-off-by: Kai Huang <[email protected]>
Signed-off-by: Simeon Widdis <[email protected]>
Signed-off-by: Chen Dai <[email protected]>
Signed-off-by: Simeon Widdis <[email protected]>
Signed-off-by: Jialiang Liang <[email protected]>
Signed-off-by: Ritvi Bhatt <[email protected]>
Signed-off-by: ritvibhatt <[email protected]>
Signed-off-by: Aaron Alvarez <[email protected]>
Signed-off-by: Aaron Alvarez <[email protected]>
Signed-off-by: Vamsi Manohar <[email protected]>
Signed-off-by: Tomoyuki Morita <[email protected]>
Signed-off-by: Lantao Jin <[email protected]>
Signed-off-by: Heng Qian <[email protected]>
Signed-off-by: ps48 <[email protected]>
Signed-off-by: Yuanchun Shen <[email protected]>
Signed-off-by: Peng Huo <[email protected]>
Signed-off-by: xinyual <[email protected]>
Co-authored-by: Kai Huang <[email protected]>
Co-authored-by: Songkan Tang <[email protected]>
Co-authored-by: Simeon Widdis <[email protected]>
Co-authored-by: Chen Dai <[email protected]>
Co-authored-by: Jialiang Liang <[email protected]>
Co-authored-by: ritvibhatt <[email protected]>
Co-authored-by: Aaron Alvarez <[email protected]>
Co-authored-by: Aaron Alvarez <[email protected]>
Co-authored-by: Vamsi Manohar <[email protected]>
Co-authored-by: Tomoyuki MORITA <[email protected]>
Co-authored-by: Lantao Jin <[email protected]>
Co-authored-by: qianheng <[email protected]>
Co-authored-by: Shenoy Pratik <[email protected]>
Co-authored-by: Yuanchun Shen <[email protected]>
Co-authored-by: Peng Huo <[email protected]>
Co-authored-by: Xinyuan Lu <[email protected]>
opensearch-trigger-bot bot pushed a commit that referenced this pull request Oct 28, 2025
* Bin command big5 queries

Signed-off-by: Kai Huang <[email protected]>

* update IT

Signed-off-by: Kai Huang <[email protected]>

* fix

Signed-off-by: Kai Huang <[email protected]>

* remove tests

Signed-off-by: Kai Huang <[email protected]>

---------

Signed-off-by: Kai Huang <[email protected]>
Signed-off-by: Kai Huang <[email protected]>
(cherry picked from commit 945235a)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
LantaoJin pushed a commit that referenced this pull request Oct 28, 2025
* Bin command big5 queries



* update IT



* fix



* remove tests



---------



(cherry picked from commit 945235a)

Signed-off-by: Kai Huang <[email protected]>
Signed-off-by: Kai Huang <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport 2.19-dev testing Related to improving software testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants