Skip to content

Pushdown DATE, TIMESTAMP, and TIMESTAMP_TZ_MILLIS predicates in MongoDB#13517

Closed
harlequin wants to merge 4 commits intotrinodb:masterfrom
harlequin:mongo-date-push
Closed

Pushdown DATE, TIMESTAMP, and TIMESTAMP_TZ_MILLIS predicates in MongoDB#13517
harlequin wants to merge 4 commits intotrinodb:masterfrom
harlequin:mongo-date-push

Conversation

@harlequin
Copy link
Copy Markdown
Contributor

Resolves: #5658
Superseeds: #5682, because it seems that the developer is not active in the moment.

This PR takes all changes as in #5682 and implement additional the proposed changes from discussion.

@cla-bot cla-bot bot added the cla-signed label Aug 5, 2022
@harlequin harlequin requested a review from ebyhr August 5, 2022 13:43
Copy link
Copy Markdown
Member

@ebyhr ebyhr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please squash commits into one. I would recommend preserving commit author when picking up other contributor's commit.

}

if (type == DATE) {
return Optional.of(new Date(TimeUnit.DAYS.toMillis((Long) trinoNativeValue)));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would recommend adding a preparatory commit to use LocalDate for DATE type in other places.

}

if (type == TIMESTAMP_MILLIS) {
return Optional.of(new Date(TimeUnit.MILLISECONDS.convert((Long) trinoNativeValue, TimeUnit.MICROSECONDS)));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use LocalDateTime

}

if (type == TIMESTAMP_TZ_MILLIS) {
return Optional.of(new Date(unpackMillisUtc((long) trinoNativeValue)));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use LocalDateTime

// bigint range, with decimal to bigint simplification
assertThat(query("SELECT regionkey, nationkey, name FROM nation WHERE nationkey BETWEEN 18.5 AND 19.5"))
.matches("VALUES (BIGINT '3', BIGINT '19', CAST('ROMANIA' AS varchar(25)))")
.isNotFullyPushedDown(FilterNode.class);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests unrelates to date, timestamp, timestamp_tz_millis change. Please separate a commit.

.addRoundTrip("time", "TIME '23:59:59.9'", createTimeType(3), "TIME '23:59:59.900'")
.addRoundTrip("time", "TIME '23:59:59.99'", createTimeType(3), "TIME '23:59:59.990'")
.addRoundTrip("time", "TIME '23:59:59.999'", createTimeType(3), "TIME '23:59:59.999'")
.addRoundTrip("time", "TIME '23:59:59.9999'", createTimeType(4), "TIME '23:59:59.9999'")
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add tests for timestamp and timestamp with time zone types.

.addRoundTrip("time", "TIME '23:59:59.999'", createTimeType(3), "TIME '23:59:59.999'")
.addRoundTrip("time", "TIME '23:59:59.9999'", createTimeType(4), "TIME '23:59:59.9999'")
.addRoundTrip("time", "TIME '23:59:59.99999'", createTimeType(5), "TIME '23:59:59.99999'")
.addRoundTrip("time", "TIME '23:59:59.999999'", createTimeType(6), "TIME '23:59:59.999999'")
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check CI failures.

@ebyhr ebyhr changed the title Pushdown DATE, TIMESTAMP, and TIMESTAMP_TZ_MILLIS predicates in mongodb connector - new attemp Pushdown DATE, TIMESTAMP, and TIMESTAMP_TZ_MILLIS predicates in MongoDB Aug 5, 2022
@yujiosaka
Copy link
Copy Markdown

hope to see some progress 🙏
i also tried a bit but it was a bit difficult for me to proceed due to the lack of the domain knowledge

@ebyhr
Copy link
Copy Markdown
Member

ebyhr commented Sep 30, 2022

Superseded by #14413

@ebyhr ebyhr closed this Sep 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

MongoDB connector timestamp predicate pushdown

3 participants