Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: preserve column qualifier for DataFrame::with_column #7792

Merged
merged 3 commits into from
Oct 11, 2023

Conversation

jonahgao
Copy link
Member

Which issue does this PR close?

Closes #7790.

Rationale for this change

The join operation produced two columns with identical names, but they belong to different relations.
We should preserve the relation names to distinguish between them.

What changes are included in this PR?

Are these changes tested?

Yes

Are there any user-facing changes?

No

@github-actions github-actions bot added the core Core DataFusion crate label Oct 11, 2023
Copy link
Contributor

@comphead comphead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also test t1 self join

@jonahgao
Copy link
Member Author

Please also test t1 self join

Added. Thank you for reviewing @comphead .

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much @jonahgao -- this looks great .

FYI @Blajda

// Test issue: https://github.com/apache/arrow-datafusion/issues/7790
// The join operation outputs two identical column names, but they belong to different relations.
#[tokio::test]
async fn with_column_join_same_columns() -> Result<()> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I verified this fails with

Error: SchemaError(AmbiguousReference { field: Column { relation: None, name: "c1" } })

Without the code change on this branch.

@alamb alamb merged commit 1bfe740 into apache:main Oct 11, 2023
22 checks passed
@jonahgao jonahgao deleted the issue_7790 branch October 11, 2023 22:20
devinjdangelo pushed a commit to devinjdangelo/arrow-datafusion that referenced this pull request Oct 11, 2023
)

* fix: preserve column qualifier for `DataFrame::with_column`

* fix test variable

* review feedback: add self join test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ambiguous reference error for named columns
3 participants