ARROW-9608: [Rust] Leaner feature gating for arrow in parquet

[vertexclique]

Currently, the parquet is installing arrow-flight and it's dependencies, which breaks the CI builds and it's unnecessary because it is not used. Parquet should work without default features of arrow by default. Simple PR will enable building it leaner.

[github-actions]

https://issues.apache.org/jira/browse/ARROW-9609

[svenwb]

FYI @andygrove @nevi-me @paddyhoran I've tested this change in our project using Rust Arrow

[sunchao]

hmm, does this mean we're removing arrow dependency from tests as well?

[vertexclique]

yes, because it is already a dependency in arrow.rs, using super::*; export in test modules will make it use the actual dependency exports of the arrow (which has been done already seems like). Tests had passed locally with no change and it passed here. I see it as good improvement tbh :)

[github-actions]

https://issues.apache.org/jira/browse/ARROW-9608

[vertexclique]

I really want to merge this PR to get rid of flight and it's dependencies while compiling parquet. My local tests and usage of this version didn't cause any problems and tests were passing. I don't know what is failing at CI at Dev / Source Release and Merge Script. Can we merge this one as it is?

[andygrove]

@vertexclique No, we'll need to fix the regression in the test caused by this change. I can help fix it. I'll look into it later today.

[andygrove]

I had a general question ... it seems inverted that arrow depends on flight. I wonder if we have the dependency the wrong way around?

[vertexclique]

Parquet crate is using arrow's default features which are default = ["flight", "prettyprint"] but flight is not needed, nor prettyprint for parquet. these are not used at all. Moreover, since features are additive, we can't override arrow dependencies features in any project that uses parquet if we continue to use default features included.
One more nit; it seems like we've added the same dependency as a dev dependency. Which is also not needed because it is in the default dependency list.

... it seems inverted that arrow depends on flight. I wonder if we have the dependency the wrong way around?

Sorry, might be I am quite a bit tired, but I didn't understand the inversion part.

[andygrove]

@vertexclique I will need to look at the code again but I don't see why arrrow would depend on a protocol. I think the protocl should depend on arrow.

The core arrow crate should have as few dependencies as possible, IMO. Same goes for Parquet.

Not everyone will be buiding distributed systems that require the protocol.

[vertexclique]

@andygrove Exactly! This is the problem we are having with the default feature gating scheme that arrow has, which scattered to arrow dependant crates.

[andygrove]

https://issues.apache.org/jira/browse/ARROW-9631

[vertexclique]

@andygrove hold on I will open a pr for that too :)

[andygrove]

@vertexclique I already did, but happy to take yours too.

#7892

[vertexclique]

@andygrove Rebased. Looking out for the CI atm.

[andygrove]

@vertexclique Just checking in on status. The PR has been idle for a while now. Should we close it?

[andygrove]

@vertexclique I'm closing this for now. Feel free to re-open if this is something that you are still working on.

[vertexclique]

@andygrove sorry for the radio silence I was on vacation. Iirc this is not needed anymore. Since we have pruned dependency tree to be leaner. Thanks!