[Backport 2.19-dev] Core Implementation of rex Command In PPL
#4239
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ject#4109) * rex - initial implementation Signed-off-by: Jialiang Liang <[email protected]> * stop using utils Signed-off-by: Jialiang Liang <[email protected]> * fix spotless check Signed-off-by: Jialiang Liang <[email protected]> * offset_field - initial implementation Signed-off-by: Jialiang Liang <[email protected]> * max_match - initial implementation Signed-off-by: Jialiang Liang <[email protected]> * sed - initial implementation Signed-off-by: Jialiang Liang <[email protected]> * fix name capture group for extraction Signed-off-by: Jialiang Liang <[email protected]> * add rex rst doc Signed-off-by: Jialiang Liang <[email protected]> * IT - initial setup Signed-off-by: Jialiang Liang <[email protected]> * add a analyzer test for legacy engine Signed-off-by: Jialiang Liang <[email protected]> * Add UT for rex Signed-off-by: Jialiang Liang <[email protected]> * sed - add pushdown for sed and explain IT and IT with fix Signed-off-by: Jialiang Liang <[email protected]> * anonymizer - add rex for anonymizer and test Signed-off-by: Jialiang Liang <[email protected]> * Add cross cluster IT for rex Signed-off-by: Jialiang Liang <[email protected]> * peng - resolve comments for rst doc 0 Signed-off-by: Jialiang Liang <[email protected]> * peng - address some comments 1 Signed-off-by: Jialiang Liang <[email protected]> * peng - resolve comment in rst doc to add a java doc link Signed-off-by: Jialiang Liang <[email protected]> * kai - modify the bin ast builder test Signed-off-by: Jialiang Liang <[email protected]> * peng - fix the extraction behavior without filter even when there is zero match Signed-off-by: Jialiang Liang <[email protected]> * fix rex explain no pushdown Signed-off-by: Jialiang Liang <[email protected]> * change the offset val output format Signed-off-by: Jialiang Liang <[email protected]> * fix rst file Signed-off-by: Jialiang Liang <[email protected]> * peng - SWITCH TO USE CALCITE NATIVE OPERATORS Signed-off-by: Jialiang Liang <[email protected]> * Peng - fix tests after operator change Signed-off-by: Jialiang Liang <[email protected]> * support mode=extract and update doc Signed-off-by: Jialiang Liang <[email protected]> * fix the issue after rebase Signed-off-by: Jialiang Liang <[email protected]> * peng - enforce specifying field in antlr for now Signed-off-by: Jialiang Liang <[email protected]> * relocate rex cmd IT Signed-off-by: Jialiang Liang <[email protected]> * peng - simplify vistFunciton Signed-off-by: Jialiang Liang <[email protected]> * peng - add UT for RexExtractMultiFunction Signed-off-by: Jialiang Liang <[email protected]> * peng - add UT RexOffsetFunction Signed-off-by: Jialiang Liang <[email protected]> * fix some tests Signed-off-by: Jialiang Liang <[email protected]> * DECOUPLE SED + OFFSET FIELD Signed-off-by: Jialiang Liang <[email protected]> * Improve error handling for extract Signed-off-by: Jialiang Liang <[email protected]> * add this rex rst into index Signed-off-by: Jialiang Liang <[email protected]> * fix return type in extract multi Signed-off-by: Jialiang Liang <[email protected]> * add rex doc into doc test Signed-off-by: Jialiang Liang <[email protected]> * fix doc test Signed-off-by: Jialiang Liang <[email protected]> * Fix linting Signed-off-by: Jialiang Liang <[email protected]> * fix rebase issue Signed-off-by: Jialiang Liang <[email protected]> * fix regex anonymizer tests Signed-off-by: Jialiang Liang <[email protected]> * fix analyzer test and setup to use util function Signed-off-by: Jialiang Liang <[email protected]> * lint fix Signed-off-by: Jialiang Liang <[email protected]> * fix doc test Signed-off-by: Jialiang Liang <[email protected]> * Add max match limit implementation Signed-off-by: Jialiang Liang <[email protected]> * fix anonymizer test Signed-off-by: Jialiang Liang <[email protected]> * peng - simplify if Signed-off-by: Jialiang Liang <[email protected]> * peng - make extract multi to only handle the case of max_match > 1 Signed-off-by: Jialiang Liang <[email protected]> --------- Signed-off-by: Jialiang Liang <[email protected]>
RyanL1997
requested review from
GumpacG,
LantaoJin,
MaxKsyunz,
Swiddis,
YANG-DB,
Yury-Fridlyand,
acarbonetto,
anirudha,
dai-chen,
derek-ho,
forestmvey,
joshuali925,
kavithacm,
mengweieric,
noCharger,
penghuo,
ps48,
qianheng-aws,
seankao-az and
ykmr1224
as code owners
ykmr1224
previously approved these changes
Sep 5, 2025
Collaborator
Author
|
I saw the doc tests failed because the sample data used for doctest is different than the one we used on main. |
Signed-off-by: Jialiang Liang <[email protected]>
Signed-off-by: Jialiang Liang <[email protected]>
dai-chen
approved these changes
Sep 9, 2025
Swiddis
approved these changes
Sep 9, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Manual backport
Related Issues
rexCommand In PPL #4109Check List
--signoffor-s.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.