Skip to content

Fix pull up lambda expression optimizer for regex_like and like function#21407

Merged
feilong-liu merged 1 commit intoprestodb:masterfrom
feilong-liu:fix_lambda_regexp
Jan 4, 2024
Merged

Fix pull up lambda expression optimizer for regex_like and like function#21407
feilong-liu merged 1 commit intoprestodb:masterfrom
feilong-liu:fix_lambda_regexp

Conversation

@feilong-liu
Copy link
Contributor

@feilong-liu feilong-liu commented Nov 16, 2023

Description

When the expression in a lambda function is of type JoniRegexType or LikePatternType (underlying Java type is Regex.class), do not extract this expression out. Instead, if it's a CAST(argument as JoniRegexType/LikePatternType), we are to extract the argument out if the argument is not of type JoniRegexType or LikePatternType

Motivation and Context

For a query like this SELECT ANY_MATCH(a, x -> (x LIKE CONCAT('a', b))) AS rejected_by_control_model, FILTER(a, x -> x LIKE CONCAT('a', b)) FROM ( SELECT ARRAY[CAST(random() AS VARCHAR)] a, CAST(random() AS VARCHAR) AS b )", "values (false, array[]), pulling out CONCAT('a', b) will fail during bytecode generation, because of the underlying Regex data type.

In this PR, I fixed it by not extracting expressions with these types.

Impact

Fix bug of the optimizer.

Test Plan

Added unit tests

Contributor checklist

  • Please make sure your submission complies with our development, formatting, commit message, and attribution guidelines.
  • PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
  • Documented new properties (with its default value), SQL syntax, functions, or other functionality.
  • If release notes are required, they follow the release notes guidelines.
  • Adequate tests were added if applicable.
  • CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

General Changes
* Fix bug of pull lambda out optimizers when expressions pulled out are of type JoniRegexType or LikePatternType

@feilong-liu feilong-liu requested a review from a team as a code owner November 16, 2023 21:59
@feilong-liu feilong-liu marked this pull request as draft November 16, 2023 21:59
@feilong-liu feilong-liu marked this pull request as ready for review November 17, 2023 00:11
@jaystarshot jaystarshot self-assigned this Nov 27, 2023
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this happen with other AbstractPrimitiveTypes which use some other java type?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update the code, instead of filter out Regex.class, use an allowlist of boolean, long, double, slice, block, which will cover most common types, including integer, bigint, varchar, map, array, boolean, row types etc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But Joni regexp is a Slice in the backend right? Just making sure that usecase still works

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, Joni regexp is has Regex as java type

super(new TypeSignature(NAME), Regex.class);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: The function can be better name, getUnderlyingRowExpressionForCast()

@pranjalssh pranjalssh changed the title Fix pull up lambda expression optimizer for reggex_like and like function Fix pull up lambda expression optimizer for regex_like and like function Jan 4, 2024
@feilong-liu feilong-liu merged commit cb582bc into prestodb:master Jan 4, 2024
@feilong-liu feilong-liu deleted the fix_lambda_regexp branch January 4, 2024 23:09
@wanglinsong wanglinsong mentioned this pull request Feb 12, 2024
64 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants