Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minor: Remove unnecessary clone in datafusion_proto #7921

Merged
merged 1 commit into from
Oct 24, 2023

Conversation

ongchi
Copy link
Contributor

@ongchi ongchi commented Oct 24, 2023

Which issue does this PR close?

Closes #.

Rationale for this change

What changes are included in this PR?

Remove unnecessary clone in LogicalPlanNode::try_into_logical_plan and PhysicalPlanNode::try_into_physical_plan.

Are these changes tested?

Are there any user-facing changes?

Comment on lines -461 to 464
let pb_partition_method = repartition.partition_method.clone().ok_or_else(|| {
let pb_partition_method = repartition.partition_method.as_ref().ok_or_else(|| {
DataFusionError::Internal(String::from(
"Protobuf deserialization error, RepartitionNode was missing required field 'partition_method'",
))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be returned early and the clone could be avoided.

Comment on lines -397 to +402
let input_schema = hash_agg
.input_schema
.as_ref()
.ok_or_else(|| {
DataFusionError::Internal(
"input_schema in AggregateNode is missing.".to_owned(),
)
})?
.clone();
let physical_schema: SchemaRef =
SchemaRef::new((&input_schema).try_into()?);
let input_schema = hash_agg.input_schema.as_ref().ok_or_else(|| {
DataFusionError::Internal(
"input_schema in AggregateNode is missing.".to_owned(),
)
})?;
let physical_schema: SchemaRef = SchemaRef::new(input_schema.try_into()?);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SchemaRef seems able create from ref of input_schema.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I is confusing, but SchemaRef is an alias for Arc<Schema> which is fast and quick to copy

Copy link
Contributor Author

@ongchi ongchi Oct 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The input_schema here is a &protobuf::Schema, and the SchemaRef is an alias of Arc<arrow_schema::Schema>.

https://github.com/apache/arrow-datafusion/blob/0911f1523ec7088bae88684ecb9bca94aa553693/datafusion/proto/src/logical_plan/from_proto.rs#L372-L379

https://github.com/apache/arrow-datafusion/blob/0911f1523ec7088bae88684ecb9bca94aa553693/datafusion/proto/src/logical_plan/from_proto.rs#L609-L620

This really confusing me at first glance, it's consume a &protobuf::Schema when calling try_into(). Seems fields will be recreated anyway, I think the clone of input_schema could be eliminated.

I don't understand why these convert trait methods convert into U from a &T rather than a T, this makes implicit clones in convert methods.

https://github.com/apache/arrow-datafusion/blob/0911f1523ec7088bae88684ecb9bca94aa553693/datafusion/proto/src/physical_plan/mod.rs#L490

And I just notice this line could be replaced by physical_schema.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree it is confusing. Any help to make it better would be most appreciated

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me -- thank you @ongchi

@alamb alamb merged commit 08c1b69 into apache:main Oct 24, 2023
22 checks passed
@ongchi ongchi deleted the proto-unnecessary-clone branch October 24, 2023 21:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants