-
Notifications
You must be signed in to change notification settings - Fork 180
[Feature] Implementation of regex Command In PPL
#4083
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature] Implementation of regex Command In PPL
#4083
Conversation
0ae9a4d to
cb2fbc7
Compare
regex Command In PPLregex Command In PPL
Signed-off-by: Jialiang Liang <[email protected]>
a9e65c7 to
5723a9f
Compare
Signed-off-by: Jialiang Liang <[email protected]>
Signed-off-by: Jialiang Liang <[email protected]>
Signed-off-by: Jialiang Liang <[email protected]>
core/src/test/java/org/opensearch/sql/analysis/AnalyzerTest.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Jialiang Liang <[email protected]>
Signed-off-by: Jialiang Liang <[email protected]>
dai-chen
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the changes!
Signed-off-by: Jialiang Liang <[email protected]>
|
The backport to To backport manually, run these commands in your terminal: # Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/sql/backport-2.19-dev 2.19-dev
# Navigate to the new working tree
pushd ../.worktrees/sql/backport-2.19-dev
# Create a new branch
git switch --create backport/backport-4083-to-2.19-dev
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 9b88f23f343c774a1b8c57f80b6e2943cb602f4f
# Push it to GitHub
git push --set-upstream origin backport/backport-4083-to-2.19-dev
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/sql/backport-2.19-devThen, create a pull request where the |
|
Will manual backport it. |
…t#4083) * implement regex cmd with calcite support by suing java library Signed-off-by: Jialiang Liang <[email protected]> * code hygiene fix Signed-off-by: Jialiang Liang <[email protected]> * comment clean up Signed-off-by: Jialiang Liang <[email protected]> * implement explain it Signed-off-by: Jialiang Liang <[email protected]> * disable regex when calcite is disable and add a test in analyzer Signed-off-by: Jialiang Liang <[email protected]> * fix spotless check Signed-off-by: Jialiang Liang <[email protected]> * [refactor] refactor some regex fn into a util class for re-usage Signed-off-by: Jialiang Liang <[email protected]> * [refactor] revert filter query builder cuz we do not need it anymore Signed-off-by: Jialiang Liang <[email protected]> * add rst docs for regex cmd Signed-off-by: Jialiang Liang <[email protected]> * add IT for regex cmd Signed-off-by: Jialiang Liang <[email protected]> * add IT for calcite no pushdown Signed-off-by: Jialiang Liang <[email protected]> * fix regex exp behavior for non string val Signed-off-by: Jialiang Liang <[email protected]> * style - remove some verbose comments Signed-off-by: Jialiang Liang <[email protected]> * remove string convertion Signed-off-by: Jialiang Liang <[email protected]> * use existing operator of REGEXP_CONTAINS Signed-off-by: Jialiang Liang <[email protected]> * fix integ test of rgex with pushdown after operator commit Signed-off-by: Jialiang Liang <[email protected]> * remove some verbose comments and fix some style Signed-off-by: Jialiang Liang <[email protected]> * fix explain it in no pushdown Signed-off-by: Jialiang Liang <[email protected]> * comment - remove unused fn for string converting Signed-off-by: Jialiang Liang <[email protected]> * remove duplicated regex match operator alias Signed-off-by: Jialiang Liang <[email protected]> * unit test - initail commit Signed-off-by: Jialiang Liang <[email protected]> * anonymizer with test Signed-off-by: Jialiang Liang <[email protected]> * fix spotlessApply Signed-off-by: Jialiang Liang <[email protected]> * add cross cluster IT Signed-off-by: Jialiang Liang <[email protected]> * fix spotless apply Signed-off-by: Jialiang Liang <[email protected]> * tomo - fix operator constant Signed-off-by: Jialiang Liang <[email protected]> * tomo - fix regex java doc Signed-off-by: Jialiang Liang <[email protected]> * tomo - field and pattern handling fix Signed-off-by: Jialiang Liang <[email protected]> * tomo - fix LRUCache Signed-off-by: Jialiang Liang <[email protected]> * tomo - remove unnecessary delegation layer Signed-off-by: Jialiang Liang <[email protected]> * rst doc fix Signed-off-by: Jialiang Liang <[email protected]> * tomo - fix comments Signed-off-by: Jialiang Liang <[email protected]> * DEFAULT FIELD related change Signed-off-by: Jialiang Liang <[email protected]> * DEFAULT FIELD - fix anonymizer tests Signed-off-by: Jialiang Liang <[email protected]> * tomo - add unit test for regex util class Signed-off-by: Jialiang Liang <[email protected]> * chen - remove code for legacy engine Signed-off-by: Jialiang Liang <[email protected]> * chen - remove stalled logic for spcified field Signed-off-by: Jialiang Liang <[email protected]> * chen - merge into 1 grammar in parser Signed-off-by: Jialiang Liang <[email protected]> * properly handle non-string field Signed-off-by: Jialiang Liang <[email protected]> * remove verbose comments Signed-off-by: Jialiang Liang <[email protected]> * remove verbose comments Signed-off-by: Jialiang Liang <[email protected]> * address commetns Signed-off-by: Jialiang Liang <[email protected]> * fix doc test for regex Signed-off-by: Jialiang Liang <[email protected]> * fix doc Signed-off-by: Jialiang Liang <[email protected]> --------- Signed-off-by: Jialiang Liang <[email protected]>
|
The backport to To backport manually, run these commands in your terminal: # Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/sql/backport-2.19-dev 2.19-dev
# Navigate to the new working tree
pushd ../.worktrees/sql/backport-2.19-dev
# Create a new branch
git switch --create backport/backport-4083-to-2.19-dev
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 9b88f23f343c774a1b8c57f80b6e2943cb602f4f
# Push it to GitHub
git push --set-upstream origin backport/backport-4083-to-2.19-dev
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/sql/backport-2.19-devThen, create a pull request where the |
|
Can you do manual backport or see what is conflicting? |
…t#4083) * implement regex cmd with calcite support by suing java library Signed-off-by: Jialiang Liang <[email protected]> * code hygiene fix Signed-off-by: Jialiang Liang <[email protected]> * comment clean up Signed-off-by: Jialiang Liang <[email protected]> * implement explain it Signed-off-by: Jialiang Liang <[email protected]> * disable regex when calcite is disable and add a test in analyzer Signed-off-by: Jialiang Liang <[email protected]> * fix spotless check Signed-off-by: Jialiang Liang <[email protected]> * [refactor] refactor some regex fn into a util class for re-usage Signed-off-by: Jialiang Liang <[email protected]> * [refactor] revert filter query builder cuz we do not need it anymore Signed-off-by: Jialiang Liang <[email protected]> * add rst docs for regex cmd Signed-off-by: Jialiang Liang <[email protected]> * add IT for regex cmd Signed-off-by: Jialiang Liang <[email protected]> * add IT for calcite no pushdown Signed-off-by: Jialiang Liang <[email protected]> * fix regex exp behavior for non string val Signed-off-by: Jialiang Liang <[email protected]> * style - remove some verbose comments Signed-off-by: Jialiang Liang <[email protected]> * remove string convertion Signed-off-by: Jialiang Liang <[email protected]> * use existing operator of REGEXP_CONTAINS Signed-off-by: Jialiang Liang <[email protected]> * fix integ test of rgex with pushdown after operator commit Signed-off-by: Jialiang Liang <[email protected]> * remove some verbose comments and fix some style Signed-off-by: Jialiang Liang <[email protected]> * fix explain it in no pushdown Signed-off-by: Jialiang Liang <[email protected]> * comment - remove unused fn for string converting Signed-off-by: Jialiang Liang <[email protected]> * remove duplicated regex match operator alias Signed-off-by: Jialiang Liang <[email protected]> * unit test - initail commit Signed-off-by: Jialiang Liang <[email protected]> * anonymizer with test Signed-off-by: Jialiang Liang <[email protected]> * fix spotlessApply Signed-off-by: Jialiang Liang <[email protected]> * add cross cluster IT Signed-off-by: Jialiang Liang <[email protected]> * fix spotless apply Signed-off-by: Jialiang Liang <[email protected]> * tomo - fix operator constant Signed-off-by: Jialiang Liang <[email protected]> * tomo - fix regex java doc Signed-off-by: Jialiang Liang <[email protected]> * tomo - field and pattern handling fix Signed-off-by: Jialiang Liang <[email protected]> * tomo - fix LRUCache Signed-off-by: Jialiang Liang <[email protected]> * tomo - remove unnecessary delegation layer Signed-off-by: Jialiang Liang <[email protected]> * rst doc fix Signed-off-by: Jialiang Liang <[email protected]> * tomo - fix comments Signed-off-by: Jialiang Liang <[email protected]> * DEFAULT FIELD related change Signed-off-by: Jialiang Liang <[email protected]> * DEFAULT FIELD - fix anonymizer tests Signed-off-by: Jialiang Liang <[email protected]> * tomo - add unit test for regex util class Signed-off-by: Jialiang Liang <[email protected]> * chen - remove code for legacy engine Signed-off-by: Jialiang Liang <[email protected]> * chen - remove stalled logic for spcified field Signed-off-by: Jialiang Liang <[email protected]> * chen - merge into 1 grammar in parser Signed-off-by: Jialiang Liang <[email protected]> * properly handle non-string field Signed-off-by: Jialiang Liang <[email protected]> * remove verbose comments Signed-off-by: Jialiang Liang <[email protected]> * remove verbose comments Signed-off-by: Jialiang Liang <[email protected]> * address commetns Signed-off-by: Jialiang Liang <[email protected]> * fix doc test for regex Signed-off-by: Jialiang Liang <[email protected]> * fix doc Signed-off-by: Jialiang Liang <[email protected]> --------- Signed-off-by: Jialiang Liang <[email protected]>
* [Feature] Implementation of `regex` Command In PPL (#4083) * implement regex cmd with calcite support by suing java library Signed-off-by: Jialiang Liang <[email protected]> * code hygiene fix Signed-off-by: Jialiang Liang <[email protected]> * comment clean up Signed-off-by: Jialiang Liang <[email protected]> * implement explain it Signed-off-by: Jialiang Liang <[email protected]> * disable regex when calcite is disable and add a test in analyzer Signed-off-by: Jialiang Liang <[email protected]> * fix spotless check Signed-off-by: Jialiang Liang <[email protected]> * [refactor] refactor some regex fn into a util class for re-usage Signed-off-by: Jialiang Liang <[email protected]> * [refactor] revert filter query builder cuz we do not need it anymore Signed-off-by: Jialiang Liang <[email protected]> * add rst docs for regex cmd Signed-off-by: Jialiang Liang <[email protected]> * add IT for regex cmd Signed-off-by: Jialiang Liang <[email protected]> * add IT for calcite no pushdown Signed-off-by: Jialiang Liang <[email protected]> * fix regex exp behavior for non string val Signed-off-by: Jialiang Liang <[email protected]> * style - remove some verbose comments Signed-off-by: Jialiang Liang <[email protected]> * remove string convertion Signed-off-by: Jialiang Liang <[email protected]> * use existing operator of REGEXP_CONTAINS Signed-off-by: Jialiang Liang <[email protected]> * fix integ test of rgex with pushdown after operator commit Signed-off-by: Jialiang Liang <[email protected]> * remove some verbose comments and fix some style Signed-off-by: Jialiang Liang <[email protected]> * fix explain it in no pushdown Signed-off-by: Jialiang Liang <[email protected]> * comment - remove unused fn for string converting Signed-off-by: Jialiang Liang <[email protected]> * remove duplicated regex match operator alias Signed-off-by: Jialiang Liang <[email protected]> * unit test - initail commit Signed-off-by: Jialiang Liang <[email protected]> * anonymizer with test Signed-off-by: Jialiang Liang <[email protected]> * fix spotlessApply Signed-off-by: Jialiang Liang <[email protected]> * add cross cluster IT Signed-off-by: Jialiang Liang <[email protected]> * fix spotless apply Signed-off-by: Jialiang Liang <[email protected]> * tomo - fix operator constant Signed-off-by: Jialiang Liang <[email protected]> * tomo - fix regex java doc Signed-off-by: Jialiang Liang <[email protected]> * tomo - field and pattern handling fix Signed-off-by: Jialiang Liang <[email protected]> * tomo - fix LRUCache Signed-off-by: Jialiang Liang <[email protected]> * tomo - remove unnecessary delegation layer Signed-off-by: Jialiang Liang <[email protected]> * rst doc fix Signed-off-by: Jialiang Liang <[email protected]> * tomo - fix comments Signed-off-by: Jialiang Liang <[email protected]> * DEFAULT FIELD related change Signed-off-by: Jialiang Liang <[email protected]> * DEFAULT FIELD - fix anonymizer tests Signed-off-by: Jialiang Liang <[email protected]> * tomo - add unit test for regex util class Signed-off-by: Jialiang Liang <[email protected]> * chen - remove code for legacy engine Signed-off-by: Jialiang Liang <[email protected]> * chen - remove stalled logic for spcified field Signed-off-by: Jialiang Liang <[email protected]> * chen - merge into 1 grammar in parser Signed-off-by: Jialiang Liang <[email protected]> * properly handle non-string field Signed-off-by: Jialiang Liang <[email protected]> * remove verbose comments Signed-off-by: Jialiang Liang <[email protected]> * remove verbose comments Signed-off-by: Jialiang Liang <[email protected]> * address commetns Signed-off-by: Jialiang Liang <[email protected]> * fix doc test for regex Signed-off-by: Jialiang Liang <[email protected]> * fix doc Signed-off-by: Jialiang Liang <[email protected]> --------- Signed-off-by: Jialiang Liang <[email protected]> * Fix merge issue (#4214) Signed-off-by: Tomoyuki Morita <[email protected]> * Fix AnalyzerTest (#4215) Signed-off-by: Tomoyuki Morita <[email protected]> --------- Signed-off-by: Jialiang Liang <[email protected]> Signed-off-by: Tomoyuki Morita <[email protected]> Co-authored-by: Tomoyuki MORITA <[email protected]>
Description
Implementation of
regexCommand In PPLDetails:
regex!=: Negated matching - returns records that do NOT match the pattern=: Positive matching - returns records that match the patternFor the handling of default field (
_source/_raw):Related Issues
regexCommand to PPL (Calcite Engine) #4082Check List
--signoff.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.