Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update rust vesion to 1.57 #1395

Merged
merged 5 commits into from
Dec 4, 2021
Merged

Conversation

xudong963
Copy link
Member

Which issue does this PR close?

Related: https://blog.rust-lang.org/2021/12/02/Rust-1.57.0.html

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

@github-actions github-actions bot added ballista datafusion Changes in the datafusion crate labels Dec 3, 2021
@houqp
Copy link
Member

houqp commented Dec 3, 2021

@xudong963 do you want to also fix the clippy errors?

@xudong963
Copy link
Member Author

@xudong963 do you want to also fix the clippy errors?

Sure 😄, a little late.

@xudong963 xudong963 force-pushed the update_rust_version branch from 4e5fcbf to 9e893dc Compare December 3, 2021 10:00
@github-actions github-actions bot added the sql SQL Planner label Dec 3, 2021
@xudong963 xudong963 force-pushed the update_rust_version branch from 9e893dc to bff569b Compare December 3, 2021 10:05
@@ -68,7 +68,7 @@ pub enum ScalarValue {
/// large binary
LargeBinary(Option<Vec<u8>>),
/// list of nested ScalarValue (boxed to reduce size_of(ScalarValue))
#[allow(clippy::box_vec)]
#[allow(clippy::box_collection)]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh clippy, how wrong you are :)

the benefit is that the overall enum is smaller (as in the size of Vec<ScalarValue> is like 3 pointers (24 bytes) but a Box<Vec<ScalarValue>> is 8 bytes

@@ -77,7 +78,10 @@ where
})?;

// first map is the iterator, second is for the `Option<_>`
array.iter().map(|x| x.map(|x| op(x)).transpose()).collect()
array
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -85,7 +85,7 @@ pub struct CreateExternalTable {
#[derive(Debug, Clone, PartialEq)]
pub enum Statement {
/// ANSI SQL AST node
Statement(SQLStatement),
Statement(Box<SQLStatement>),
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xudong963 xudong963 force-pushed the update_rust_version branch from bff569b to 30dd9c9 Compare December 3, 2021 10:34
@xudong963
Copy link
Member Author

I can't find field is never read clippy update in https://github.com/rust-lang/rust-clippy/blob/master/CHANGELOG.md#rust-157

8KxCTexjiN

So I still don't fix clippy errors in the screenshot.

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @xudong963 -- there are still a few clippy failures https://github.com/apache/arrow-datafusion/runs/4406965470?check_suite_focus=true

They mostly look related to "fields not read" and some of that is surprising to me (like perhaps we have some missing functionality that needs to be fleshed out).

One approach we might take is add clippy allow lints and then go work through them one by one. I'll try and do this later today if I get to it

@@ -68,7 +68,7 @@ pub enum ScalarValue {
/// large binary
LargeBinary(Option<Vec<u8>>),
/// list of nested ScalarValue (boxed to reduce size_of(ScalarValue))
#[allow(clippy::box_vec)]
#[allow(clippy::box_collection)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh clippy, how wrong you are :)

the benefit is that the overall enum is smaller (as in the size of Vec<ScalarValue> is like 3 pointers (24 bytes) but a Box<Vec<ScalarValue>> is 8 bytes

@xudong963
Copy link
Member Author

One approach we might take is add clippy allow lints and then go work through them one by one. I'll try and do this later today if I get to it

You mean add #[allow(dead_code)]?

@alamb
Copy link
Contributor

alamb commented Dec 3, 2021

@xudong963 I pushed a few changes to this branch to try and get clippy to pass. Here's hoping we get a clean run

@@ -37,8 +37,8 @@ use super::{format_state_name, sum};
#[derive(Debug)]
pub struct Avg {
name: String,
#[allow(dead_code)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will look into removing this field as well

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR might be the reason causes the different behavior on dead_code.
rust-lang/rust#85200

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are right

@@ -346,7 +346,7 @@ struct SortPreservingMergeStream {
receivers: Vec<mpsc::Receiver<ArrowResult<RecordBatch>>>,

/// Drop helper for tasks feeding the [`receivers`](Self::receivers)
drop_helper: AbortOnDropMany<()>,
_drop_helper: AbortOnDropMany<()>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is used (its drop() impl is used) so change its name

@@ -31,22 +31,22 @@ pub const BALLISTA_DEFAULT_SHUFFLE_PARTITIONS: &str = "ballista.shuffle.partitio
#[derive(Debug, Clone)]
pub struct ConfigEntry {
name: String,
description: String,
data_type: DataType,
_description: String,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't know if this was important or not to keep in ballista so rather than removing it I renamed them to start with _ to satisfy clippy

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, very much agree with you, _ is a conservative but useful way!

}

impl BuiltInWindowExpr {
/// create a new built-in window function expression
pub(super) fn new(
fun: BuiltInWindowFunction,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the fun and window_frame are used to construct the BuiltInWindowFunctionExpr (so the don't need to be also encoded on the BuildInWindowExpr

#[structopt(short = "s", long = "batch-size", default_value = "8192")]
batch_size: usize,

// /// Batch size when reading CSV or Parquet files
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These parameters are never read (aka they don't do anything) so removing them from the CLI seemed like a reasonable thing to do

@xudong963
Copy link
Member Author

Nice, clippy passed😄

// Average is always Float64, but Avg::new() has a data_type
// parameter to keep a consistent signature with the other
// Aggregate expressions.
assert_eq!(data_type, DataType::Float64);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@alamb alamb merged commit d047900 into apache:master Dec 4, 2021
@alamb
Copy link
Contributor

alamb commented Dec 4, 2021

Thanks again @xudong963.

@xudong963 xudong963 deleted the update_rust_version branch December 4, 2021 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datafusion Changes in the datafusion crate sql SQL Planner
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants