Skip to content

Conversation

@findepi
Copy link
Member

@findepi findepi commented Aug 7, 2025

ScalarUDFImpl::is_nullable is deprecated since d3f1c9a. The ScalarUDF's wrapper also needs deprecation to allow eventually removing the deprecated ScalarUDFImpl::is_nullable.

cc @Blizzara

`ScalarUDFImpl::is_nullable` is deprecated. The `ScalarUDF`'s wrapper
also needs deprecation to allow eventually removing the deprecated
`ScalarUDFImpl::is_nullable`.
@findepi findepi requested a review from jayzhan211 August 7, 2025 15:25
@github-actions github-actions bot added the logical-expr Logical plan and expressions label Aug 7, 2025
@findepi findepi requested a review from xudong963 August 7, 2025 20:24
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

THANK YOU

@Blizzara
Copy link
Contributor

Blizzara commented Aug 7, 2025

Thanks! The deprecation is sure an improvement, but I wonder if overall this is_nullable -> return type from args change suffers from the same issue as the earlier ones, namely that if a user implements a custom UDF overriding the is_nullable, they don’t see the deprecation warning and so their compiler won’t likely warn them to switch to the new one. If I remember correctly, the ”best” way I could figure out for these cases where

  • a new function is introduced in the (Scalar)UdfImpl interface and switched to internally, and
  • the new function does not by default delegate to the previously-internally-used function,
    would be to just remove the old function completely from the interface. Otherwise downstream implementors won’t get compile-time warnings, which is unfortunate. Obviously full removal has its own downsides too…

(I may be also a bit rusty in my thinking, haven’t been working on this for a while anymore now.)

@findepi findepi merged commit 060938b into apache:main Aug 8, 2025
27 checks passed
@findepi findepi deleted the findepi/deprecate-scalarudf-is-nullable-7b1a30 branch August 8, 2025 05:57
@findepi
Copy link
Member Author

findepi commented Aug 8, 2025

the new function does not by default delegate to the previously-internally-used function

Typically the new function delegates to the old one and the old one gets deprecated.

If that's not possible, adding such new function is a breaking API change. In such case, graceful deprecation of the old function doesn't help, and actually makes situation worse, as you said -- no compiler's help.

Given exactly this was the case with ScalarUDFImpl::is_nullable in d3f1c9a, I suggest removal of the is_nullable function. However, I don't have full context and I am not the author of that change.
@Blizzara do you want to make a follow-on PR with the removal?

@alamb
Copy link
Contributor

alamb commented Aug 8, 2025

I agree that simply removing the is_nullable method, even though it is a breaking API, might be the least confusing course of action

now that we have an Upgrade Guide, we have a place to provide more context for these changes too so I think thier impact is less jarring / seemingly arbitrary

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

logical-expr Logical plan and expressions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants