-
Notifications
You must be signed in to change notification settings - Fork 1.1k
docs: Improve main README.md and highlight community #9119
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -19,23 +19,49 @@ | |
|
|
||
| # Native Rust implementation of Apache Arrow and Apache Parquet | ||
|
|
||
| Welcome to the [Rust][rust] implementation of [Apache Arrow], the popular in-memory columnar format. | ||
| Welcome to the [Rust][rust] implementation of [Apache Arrow], a popular | ||
| in-memory columnar format and [Apache Parquet], a popular columnar file | ||
| format. | ||
|
|
||
| ## Community | ||
|
|
||
| We welcome participation from everyone and encourage you to join us, ask | ||
| questions, help others, and get involved. All participation in the Apache Arrow | ||
| project is governed by the Apache Software Foundation's [code of | ||
| conduct](https://www.apache.org/foundation/policies/conduct.html). | ||
|
|
||
| We use GitHub [issues] and [pull requests] for all technical discussions, reviews, | ||
| new features, bug fixes and release coordination. This ensures that all communication | ||
| is public and archived for future reference. | ||
|
|
||
| The `[email protected]` mailing list is the communication channel for the overall Apache Arrow community. | ||
| Instructions for signing up and links to the archives can be found on the [Arrow Community](https://arrow.apache.org/community/) page. | ||
|
|
||
| Some community members also use the [Arrow Rust Discord Server](https://discord.gg/YAb2TdazKQ) and the official [ASF Slack](https://s.apache.org/slack-invite) server for informal discussions and coordination. | ||
| This is a great place to meet other contributors and get guidance on where to contribute. | ||
| However, all technical designs should also be recorded and formalized in GitHub issues, so that they are accessible to everyone. | ||
| In Slack, find us in the `#arrow-rust` channel and feel free to ask for an invite via Discord, GitHub issues, or other means. | ||
|
|
||
| There is more information in the [contributing] guide. | ||
|
|
||
| ## Repository Structure | ||
|
|
||
| This repository contains the following crates: | ||
|
|
||
| | Crate | Description | Latest API Docs | README | | ||
| | ------------------ | ---------------------------------------------------------------------------- | ------------------------------------------------ | --------------------------------- | | ||
| | [`arrow`] | Core functionality (memory layout, arrays, low level computations) | [docs.rs](https://docs.rs/arrow/latest) | [(README)][arrow-readme] | | ||
| | [`arrow-flight`] | Support for Arrow-Flight IPC protocol | [docs.rs](https://docs.rs/arrow-flight/latest) | [(README)][flight-readme] | | ||
| | [`parquet`] | Support for Parquet columnar file format | [docs.rs](https://docs.rs/parquet/latest) | [(README)][parquet-readme] | | ||
| | [`parquet`] | Support for the [Apache Parquet] columnar file format | [docs.rs](https://docs.rs/parquet/latest) | [(README)][parquet-readme] | | ||
| | [`parquet_derive`] | A crate for deriving RecordWriter/RecordReader for arbitrary, simple structs | [docs.rs](https://docs.rs/parquet-derive/latest) | [(README)][parquet-derive-readme] | | ||
|
|
||
| The current development version the API documentation in this repo can be found [here](https://arrow.apache.org/rust). | ||
| The current development version the API documentation can be found [here](https://arrow.apache.org/rust). | ||
|
|
||
| Note: previously the [`object_store`] crate was also part of this repository, | ||
| but it has been moved to the [arrow-rs-object-store repository] | ||
|
|
||
| [apache arrow]: https://arrow.apache.org/ | ||
| [apache parquet]: https://parquet.apache.org/ | ||
| [`arrow`]: https://crates.io/crates/arrow | ||
| [`parquet`]: https://crates.io/crates/parquet | ||
| [`parquet_derive`]: https://crates.io/crates/parquet-derive | ||
|
|
@@ -49,7 +75,7 @@ Versioning]. | |
|
|
||
| Due to available maintainer and testing bandwidth, [`arrow`] crates ([`arrow`], | ||
| [`arrow-flight`], etc.) are released on the same schedule with the same versions | ||
| as the [`parquet`] and [`parquet-derive`] crates. | ||
| as the [`parquet`] and [`parquet_derive`] crates. | ||
|
|
||
| This crate releases every month. We release new major versions (with potentially | ||
| breaking API changes) at most once a quarter, and release incremental minor | ||
|
|
@@ -81,7 +107,7 @@ Planned Release Schedule | |
|
|
||
| ### Rust Version Compatibility Policy | ||
|
|
||
| arrow-rs and parquet are built and tested with stable Rust, and will keep a rolling MSRV (minimum supported Rust version) that can only be updated in major releases on a need by basis (e.g. project dependencies bump their MSRV or a particular Rust feature is useful for us etc.). The new MSRV if selected will be at least 6 months old. The minor releases are guaranteed to have the same MSRV. | ||
| arrow-rs and parquet are built and tested with stable Rust, and will keep a rolling MSRV (minimum supported Rust version) that can only be updated in major releases on an as needed basis (e.g. project dependencies bump their MSRV or a particular Rust feature is useful for us etc.). The new MSRV if selected will be at least 6 months old. The minor releases are guaranteed to have the same MSRV. | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. driveby wordsmithing |
||
|
|
||
| Note: If a Rust hotfix is released for the current MSRV, the MSRV will be updated to the specific minor version that includes all applicable hotfixes preceding other policies. | ||
|
|
||
|
|
@@ -152,24 +178,6 @@ including `join`s and window functions. | |
|
|
||
| You can find more details about each crate in their respective READMEs. | ||
|
|
||
| ## Arrow Rust Community | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. moved to the top and clarified |
||
|
|
||
| The `[email protected]` mailing list serves as the core communication channel for the Arrow community. Instructions for signing up and links to the archives can be found on the [Arrow Community](https://arrow.apache.org/community/) page. All major announcements and communications happen there. | ||
|
|
||
| The Rust Arrow community also uses the official [ASF Slack](https://s.apache.org/slack-invite) for informal discussions and coordination. This is | ||
| a great place to meet other contributors and get guidance on where to contribute. Join us in the `#arrow-rust` channel and feel free to ask for an invite via: | ||
|
|
||
| 1. the `[email protected]` mailing list | ||
| 2. the [GitHub Discussions][discussions] | ||
| 3. the [Discord channel](https://discord.gg/YAb2TdazKQ) | ||
|
|
||
| The Rust implementation uses [GitHub issues][issues] as the system of record for new features and bug fixes and | ||
| this plays a critical role in the release process. | ||
|
|
||
| For design discussions we generally use GitHub issues. | ||
|
|
||
| There is more information in the [contributing] guide. | ||
|
|
||
| [rust]: https://www.rust-lang.org/ | ||
| [`object_store`]: https://crates.io/crates/object-store | ||
| [arrow-readme]: arrow/README.md | ||
|
|
@@ -180,4 +188,5 @@ There is more information in the [contributing] guide. | |
| [ballista-readme]: https://github.com/apache/datafusion-ballista/blob/main/README.md | ||
| [parquet-derive-readme]: parquet_derive/README.md | ||
| [issues]: https://github.com/apache/arrow-rs/issues | ||
| [pull requests]: https://github.com/apache/arrow-rs/pulls | ||
| [discussions]: https://github.com/apache/arrow-rs/discussions | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also note this repo has the parquet implementation