-
Notifications
You must be signed in to change notification settings - Fork 219
Support bin command with Calcite
#3878
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 81 commits
Commits
Show all changes
82 commits
Select commit
Hold shift + click to select a range
a141c22
Support bin command with Calcite
ahkcs 36d3d2c
add .gitignore
ahkcs 366ff5f
Remove settings file
ahkcs 0426092
add performance testing
ahkcs bfce28c
Fix CI
ahkcs 9a79063
fix style
ahkcs 6ef78c4
support minspan
ahkcs 7d0e75b
support aligntime
ahkcs 47b6764
fix
ahkcs 52f55dd
fix CI
ahkcs 410c088
refactor
ahkcs 39525bd
refactor
ahkcs a664bd2
formatting
ahkcs ead4169
fix CI
ahkcs 6e2d5ec
Big Bin commands Modification
ahkcs f5d60a3
fix CI
ahkcs 727c9b2
fix syntax
ahkcs be3b123
Major change in Bin command
ahkcs bcf5408
update CalciteExplainIT
ahkcs 3c79545
fix range string parsing
ahkcs 7145101
fix aligntime
ahkcs 2fae14c
fix subseconds
ahkcs 367031a
IT change
ahkcs 1e00bc4
fix IT
ahkcs 4e731b8
update explain IT
ahkcs 2cc7d36
update bin.rst
ahkcs 6370d0c
fix IT
ahkcs a86cd6d
delete unnecessary code
ahkcs a55e93e
fix CI
ahkcs edfaf6f
fix bin aggregation
ahkcs 7f2a874
fix IT
ahkcs 71873b4
fixes
ahkcs 8478df3
fix
ahkcs e1824de
add thorough IT tests
ahkcs ff7447d
add support for log
ahkcs 04bffcd
remove
ahkcs 66e33a0
fix
ahkcs acc9d94
wraps the bins parameter logic
ahkcs 47bd4ad
TIMESTAMP and flexible order of command options
ahkcs fd8d488
fix CI
ahkcs 4eced40
optimize plan
ahkcs dd711e4
renaming
ahkcs 70ce83b
use mathematic approach for nice_width
ahkcs 230696f
fix doc and log
ahkcs 5a988cc
refactored the Bin class into an abstract class
ahkcs 9c8f749
Refactoring
ahkcs dba465b
fix CI
ahkcs b749400
rebase and fix CI
ahkcs 44bd865
fixes
ahkcs a2c040d
fix aggregation issue
ahkcs 9d56ada
style fixes
ahkcs aebfb2d
remove changes on datetime functions
ahkcs b26379f
revert changes
ahkcs d6d036f
put files back
ahkcs 3815fc0
fixes
ahkcs fc11a62
fix
ahkcs a936353
bin.rst update
ahkcs 4db6506
remove
ahkcs f988430
Add to CrossClusterSearchIT
ahkcs 26debf1
Add CrossClusterBinCommandIT
ahkcs c0a1bba
enable calcite
ahkcs f2c3401
Fix
ahkcs 0b2d843
Fix CI
ahkcs c7bd21a
fixes
ahkcs 4d81ccd
fix explain
ahkcs 350eaf4
Fixes
ahkcs c844af2
remove
ahkcs e3a765e
update
ahkcs 6f7ae56
Add UT
ahkcs 598e84a
fix CI
ahkcs 7baf3f1
modify bin.rst
ahkcs f2232fb
modify path
ahkcs cd3b499
update doc and redundancy
ahkcs 679368c
move cross cluster IT
ahkcs 183dbdc
Merge branch 'main' into feat/bin_command
ahkcs 4079fdf
fix CI
ahkcs 5731736
separate aggregation
ahkcs b2fbe70
remove unused test
ahkcs 6f4c801
fixes
ahkcs 2291506
fixes
ahkcs 10dd000
update bin.rst
ahkcs 71ee3d3
fixes
ahkcs File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| /* | ||
| * Copyright OpenSearch Contributors | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package org.opensearch.sql.ast.tree; | ||
|
|
||
| import com.google.common.collect.ImmutableList; | ||
| import java.util.List; | ||
| import java.util.Optional; | ||
| import lombok.EqualsAndHashCode; | ||
| import lombok.Getter; | ||
| import lombok.Setter; | ||
| import lombok.ToString; | ||
| import org.opensearch.sql.ast.AbstractNodeVisitor; | ||
| import org.opensearch.sql.ast.expression.UnresolvedExpression; | ||
|
|
||
| /** Abstract AST node representing Bin operations with type-safe derived classes. */ | ||
| @Getter | ||
| @Setter | ||
| @ToString | ||
| @EqualsAndHashCode(callSuper = false) | ||
| public abstract class Bin extends UnresolvedPlan { | ||
|
|
||
| private UnresolvedPlan child; | ||
|
|
||
| protected final UnresolvedExpression field; | ||
|
|
||
| protected final Optional<String> alias; | ||
|
|
||
| protected Bin(UnresolvedExpression field, Optional<String> alias) { | ||
| this.field = field; | ||
| this.alias = alias; | ||
| } | ||
|
|
||
| /** | ||
| * Validates the parameters specific to this bin type. Each subclass implements its own validation | ||
| * logic. | ||
| */ | ||
| public abstract void validate(); | ||
|
|
||
| @Override | ||
| public Bin attach(UnresolvedPlan child) { | ||
| this.child = child; | ||
| return this; | ||
| } | ||
|
|
||
| @Override | ||
| public List<UnresolvedPlan> getChild() { | ||
| return this.child == null ? ImmutableList.of() : ImmutableList.of(this.child); | ||
| } | ||
|
|
||
| @Override | ||
| public <T, C> T accept(AbstractNodeVisitor<T, C> nodeVisitor, C context) { | ||
| return nodeVisitor.visitBin(this, context); | ||
| } | ||
| } |
55 changes: 55 additions & 0 deletions
55
core/src/main/java/org/opensearch/sql/ast/tree/CountBin.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| /* | ||
| * Copyright OpenSearch Contributors | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package org.opensearch.sql.ast.tree; | ||
|
|
||
| import java.util.Optional; | ||
| import lombok.Builder; | ||
| import lombok.EqualsAndHashCode; | ||
| import lombok.Getter; | ||
| import lombok.ToString; | ||
| import org.opensearch.sql.ast.expression.UnresolvedExpression; | ||
| import org.opensearch.sql.calcite.utils.binning.BinConstants; | ||
|
|
||
| /** | ||
| * AST node representing count-based bin operation. This is the third priority bin type that uses | ||
| * "nice number" algorithm to create a specific number of bins. Supports start/end range parameters. | ||
| */ | ||
| @Getter | ||
| @ToString(callSuper = true) | ||
| @EqualsAndHashCode(callSuper = true) | ||
| public class CountBin extends Bin { | ||
|
|
||
| private final Integer bins; | ||
|
|
||
| private final Optional<UnresolvedExpression> start; | ||
|
|
||
| private final Optional<UnresolvedExpression> end; | ||
|
|
||
| @Builder | ||
| public CountBin( | ||
| UnresolvedExpression field, | ||
| Optional<String> alias, | ||
| Integer bins, | ||
| Optional<UnresolvedExpression> start, | ||
| Optional<UnresolvedExpression> end) { | ||
| super(field, alias); | ||
| this.bins = bins; | ||
| this.start = start; | ||
| this.end = end; | ||
| validate(); | ||
| } | ||
|
|
||
| @Override | ||
| public void validate() { | ||
| // Bins count validation based on documentation | ||
| if (bins < BinConstants.MIN_BINS || bins > BinConstants.MAX_BINS) { | ||
| throw new IllegalArgumentException( | ||
| String.format( | ||
| "The bins parameter must be between %d and %d, got: %d", | ||
| BinConstants.MIN_BINS, BinConstants.MAX_BINS, bins)); | ||
| } | ||
| } | ||
| } |
35 changes: 35 additions & 0 deletions
35
core/src/main/java/org/opensearch/sql/ast/tree/DefaultBin.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| /* | ||
| * Copyright OpenSearch Contributors | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package org.opensearch.sql.ast.tree; | ||
|
|
||
| import java.util.Optional; | ||
| import lombok.Builder; | ||
| import lombok.EqualsAndHashCode; | ||
| import lombok.Getter; | ||
| import lombok.ToString; | ||
| import org.opensearch.sql.ast.expression.UnresolvedExpression; | ||
|
|
||
| /** | ||
| * AST node representing default magnitude-based bin operation. This is the lowest priority bin type | ||
| * that uses automatic magnitude-based algorithm when no explicit binning parameters are specified. | ||
| */ | ||
| @Getter | ||
| @ToString(callSuper = true) | ||
| @EqualsAndHashCode(callSuper = true) | ||
| public class DefaultBin extends Bin { | ||
|
|
||
| @Builder | ||
| public DefaultBin(UnresolvedExpression field, Optional<String> alias) { | ||
| super(field, alias); | ||
| validate(); | ||
| } | ||
|
|
||
| @Override | ||
| public void validate() { | ||
| // Default bin has no additional parameters to validate | ||
| // Field validation is already handled in the base class | ||
| } | ||
| } |
46 changes: 46 additions & 0 deletions
46
core/src/main/java/org/opensearch/sql/ast/tree/MinSpanBin.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| /* | ||
| * Copyright OpenSearch Contributors | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package org.opensearch.sql.ast.tree; | ||
|
|
||
| import java.util.Optional; | ||
| import lombok.Builder; | ||
| import lombok.EqualsAndHashCode; | ||
| import lombok.Getter; | ||
| import lombok.ToString; | ||
| import org.opensearch.sql.ast.expression.UnresolvedExpression; | ||
|
|
||
| /** | ||
| * AST node representing minimum span-based bin operation. This is the second priority bin type that | ||
| * uses magnitude-based algorithm with minimum span constraint. Supports start/end range parameters. | ||
| */ | ||
| @Getter | ||
| @ToString(callSuper = true) | ||
| @EqualsAndHashCode(callSuper = true) | ||
| public class MinSpanBin extends Bin { | ||
|
|
||
| private final UnresolvedExpression minspan; | ||
|
|
||
| private final Optional<UnresolvedExpression> start; | ||
|
|
||
| private final Optional<UnresolvedExpression> end; | ||
|
|
||
| @Builder | ||
| public MinSpanBin( | ||
| UnresolvedExpression field, | ||
| Optional<String> alias, | ||
| UnresolvedExpression minspan, | ||
| Optional<UnresolvedExpression> start, | ||
| Optional<UnresolvedExpression> end) { | ||
| super(field, alias); | ||
| this.minspan = minspan; | ||
| this.start = start; | ||
| this.end = end; | ||
| validate(); | ||
| } | ||
|
|
||
| @Override | ||
| public void validate() {} | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I doubt whether changes to
AstDSL.javais necessary since you have marked bin command as only available when Calcite is enabled.AstDSL.javais mainly used to build expression AST for v2. (Please correct me if I am wrong)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is AST layer. We can keep this API.