Skip to content

marvinlanhenke/iceberg-rust

 
 

Repository files navigation

Apache Iceberg Rust

Native Rust implementation of Apache Iceberg.

Roadmap

Catalog

Catalog Type Status
Rest Done
Hive Done
Sql In Progress
Glue Done
DynamoDB Not Started

FileIO

FileIO Type Status
S3 Done
Local File Done
GCS Not Started
HDFS Not Started

Our FileIO is powered by Apache OpenDAL, so it would be quite easy to expand to other service.

Table API

Reader

Feature Status
File based task planning Done
Size based task planning Not started
Filter pushdown(manifest evaluation, partition prunning) In Progress
Apply deletions, including equality and position deletions Not started
Read into arrow record batch In Progress
Parquet file support Done
ORC file support Not started

Writer

Feature Status
Data writer Not started
Equality deletion writer Not started
Position deletion writer Not started
Partitioned writer Not started
Upsert writer Not started
Parquet file support Not started
ORC file support Not started

Transaction

Feature Status
Schema evolution Not started
Update partition spec Not started
Update properties Not started
Replace sort order Not started
Update location Not started
Append files Not started
Rewrite files Not started
Rewrite manifests Not started
Overwrite files Not started
Row level updates Not started
Replace partitions Not started
Snapshot management Not started

Integrations

We will add integrations with other rust based data systems, such as polars, datafusion, etc.

Contribute

Iceberg is an active open-source project. We are always open to people who want to use it or contribute to it. Here are some ways to go.

License

Licensed under the Apache License, Version 2.0

Releases

No releases published

Packages

No packages published

Languages

  • Rust 99.5%
  • Other 0.5%