chore: upgrade DataFusion family, again#7578
Conversation
|
Important Installation incomplete: to start using Gemini Code Assist, please ask the organization owner(s) to visit the Gemini Code Assist Admin Console and sign the Terms of Services. |
github-actions
Bot
added
size/L
docs-not-required
MichaelScofield
force-pushed
the
chore/upgrade-datafusion-family
branch
2 times, most recently
from
8826f69 to
f1a72f2
Compare
MichaelScofield
force-pushed
the
chore/upgrade-datafusion-family
branch
8 times, most recently
from
5d8d1ec to
fe569bb
Compare
MichaelScofield
requested review from
a team,
discord9,
evenyag,
v0y4g3r and
waynexia
as code owners
|
@codex review |
|
@gemini-code-assist review |
There was a problem hiding this comment.
Pull request overview
This PR upgrades the DataFusion family of crates to their latest versions and updates sqlparser to version 0.60.0, enabling adoption of important new features.
Changes:
- Updated parser code to accommodate API changes in the new sqlparser version
- Test results updated to reflect query plan optimizations in the new DataFusion version
- Query plans now exclude certain intermediate operators (CoalesceBatchesExec, CooperativeExec in some contexts)
Reviewed changes
Copilot reviewed 108 out of 110 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/sql/src/parsers/with_tql_parser.rs | Added explicit: false field to TableAlias construction for API compatibility |
| src/sql/src/parsers/insert_parser.rs | Updated parse_insert calls to pass token parameter as required by new API |
| tests/cases//.result | Updated test results reflecting DataFusion query plan optimizations and output format changes |
| tests/cases//.sql | Added quotes to column names that are now reserved keywords |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request upgrades DataFusion to its latest version, along with sqlparser and other related dependencies. The code changes primarily adapt to the new APIs from these upgraded libraries. The changes appear to be correct and handle the API breakages well. However, I've noticed a potential regression in the behavior of the ROUND function for DECIMAL types in the new DataFusion version, as reflected in the updated test results. This might warrant further investigation.
|
Codex Review: Didn't find any major issues. Already looking forward to the next diff. ℹ️ About Codex in GitHubCodex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback". |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 132 out of 134 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cc82714201
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
MichaelScofield
force-pushed
the
chore/upgrade-datafusion-family
branch
2 times, most recently
from
bc03cec to
3c4a4c3
Compare
MichaelScofield
force-pushed
the
chore/upgrade-datafusion-family
branch
from
3c4a4c3 to
d13dcd9
Compare
Signed-off-by: luofucong <luofc@foxmail.com>
MichaelScofield
force-pushed
the
chore/upgrade-datafusion-family
branch
from
d13dcd9 to
00b3bcf
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 132 out of 134 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
What's changed and what's your intention?
Upgrade DataFusion to the commit just before using Arrow 58; and sqlparser to v0.61.0
PR Checklist
Please convert it to a draft if some of the following conditions are not met.