Skip to content

Conversation

@ebyhr
Copy link
Member

@ebyhr ebyhr commented Dec 6, 2022

Description

Fixes #14078

Release notes

(x) Release notes are required, with the following suggested text:

# General
* Improve performance of queries that contain predicates involving `year` function. ({issue}`14078`)

@cla-bot cla-bot bot added the cla-signed label Dec 6, 2022
@ebyhr ebyhr force-pushed the ebi/iceberg-year-function branch from c7dd8ca to 206a236 Compare December 6, 2022 03:14
@ebyhr ebyhr self-assigned this Dec 6, 2022
@ebyhr ebyhr marked this pull request as ready for review December 7, 2022 10:21
@ebyhr ebyhr requested a review from findepi December 7, 2022 10:29
@ebyhr ebyhr force-pushed the ebi/iceberg-year-function branch from 206a236 to 43f5af3 Compare December 8, 2022 01:09
@ebyhr
Copy link
Member Author

ebyhr commented Dec 8, 2022

Addressed comments.

@ebyhr ebyhr force-pushed the ebi/iceberg-year-function branch from 43f5af3 to ada8cd3 Compare December 10, 2022 00:53
@findinpath
Copy link
Contributor

CI hit #13633

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I correct it's not possible to normalize year() to date_trunc('year' or the other way around?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the value is known to be long (or int), let the caller do the conversion.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.with(firstDayOfYear()) seems redundant

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not LocalDateTime.of(year, 1, 1, 0, 0)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LocalDateTime.of(year, 1, 1, 0, 0)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (argumentType != DATE && !(argumentType instanceof TimestampType)) {
// e.g. year(INTERVAL) not handled here
return expression;
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I understand the comment.

per the comment below (// if types do no match, this cannot be a comparison function) we didn't verify yet the call represents a comparison, so we don't know anything about relation between firstArgument and secondArgument

let's remove the comment

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LocalTime.MIN looks better in this context or perhaps

int year = toIntExact((Long) constant.getValue());
ZonedDateTime periodStart = ZonedDateTime.of(year, 1, 1, 0, 0, 0, 0, UTC);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right-open ranges are easier to construct and work with, see unwrapTimestampTzToDateCast

in fact, there is high overlap between that method and the new one, opportunity for re-use

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The TODO got removed, but this still reads .isNotFullyPushedDown(); instead of .isFullyPushedDown().
It looks like the test asserts that the functionality doesn't work as intended.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw #16017

@ebyhr ebyhr force-pushed the ebi/iceberg-year-function branch from ada8cd3 to 474cb1c Compare February 13, 2023 09:51
@ebyhr ebyhr force-pushed the ebi/iceberg-year-function branch from 474cb1c to ae62119 Compare February 13, 2023 11:38
@ebyhr ebyhr closed this Feb 14, 2023
@ebyhr ebyhr deleted the ebi/iceberg-year-function branch February 14, 2023 00:11
@ebyhr
Copy link
Member Author

ebyhr commented Feb 14, 2023

I accidentally deleted the branch. Sent a new PR #16106

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

Iceberg predicate pushdown for date/time columns and year() function

3 participants