Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions docs/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
---
title: "Introduction"
menu: main
weight: 0
---
<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
-->

![Iceberg](./img/Iceberg-logo.png)

**Apache Iceberg is an open table format for huge analytic datasets.** Iceberg adds tables to compute engines including Spark, Trino, PrestoDB, Flink and Hive using a high-performance table format that works just like a SQL table.

### User experience

Iceberg avoids unpleasant surprises. Schema evolution works and won't inadvertently un-delete data. Users don't need to know about partitioning to get fast queries.

* [Schema evolution](evolution#schema-evolution) supports add, drop, update, or rename, and has [no side-effects](evolution#correctness)
* [Hidden partitioning](partitioning) prevents user mistakes that cause silently incorrect results or extremely slow queries
* [Partition layout evolution](evolution#partition-evolution) can update the layout of a table as data volume or query patterns change
* [Time travel](spark-queries#time-travel) enables reproducible queries that use exactly the same table snapshot, or lets users easily examine changes
* Version rollback allows users to quickly correct problems by resetting tables to a good state

### Reliability and performance

Iceberg was built for huge tables. Iceberg is used in production where a single table can contain tens of petabytes of data and even these huge tables can be read without a distributed SQL engine.

* [Scan planning is fast](performance#scan-planning) -- a distributed SQL engine isn't needed to read a table or find files
* [Advanced filtering](performance#data-filtering) -- data files are pruned with partition and column-level stats, using table metadata

Iceberg was designed to solve correctness problems in eventually-consistent cloud object stores.

* [Works with any cloud store](reliability) and reduces NN congestion when in HDFS, by avoiding listing and renames
* [Serializable isolation](reliability) -- table changes are atomic and readers never see partial or uncommitted changes
* [Multiple concurrent writers](reliability#concurrent-write-operations) use optimistic concurrency and will retry to ensure that compatible updates succeed, even when writes conflict

### Open standard

Iceberg has been designed and developed to be an open community standard with a [specification](../../spec) to ensure compatibility across languages and implementations.

[Apache Iceberg is open source](../../community), and is developed at the [Apache Software Foundation](https://www.apache.org/).

5 changes: 0 additions & 5 deletions docs/api/_index.md

This file was deleted.

5 changes: 0 additions & 5 deletions docs/api/javadoc/_index.md

This file was deleted.

6 changes: 0 additions & 6 deletions docs/asf/_index.md

This file was deleted.

22 changes: 0 additions & 22 deletions docs/asf/donate/_index.md

This file was deleted.

22 changes: 0 additions & 22 deletions docs/asf/events/_index.md

This file was deleted.

22 changes: 0 additions & 22 deletions docs/asf/license/_index.md

This file was deleted.

22 changes: 0 additions & 22 deletions docs/asf/security/_index.md

This file was deleted.

22 changes: 0 additions & 22 deletions docs/asf/sponsors/_index.md

This file was deleted.

23 changes: 0 additions & 23 deletions docs/athena/_index.md

This file was deleted.

4 changes: 4 additions & 0 deletions docs/integrations/aws.md → docs/aws.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
---
title: "AWS"
url: aws
menu:
main:
parent: Integrations
weight: 0
---
<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
Expand Down
7 changes: 5 additions & 2 deletions docs/community/blogs.md → docs/blogs.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
---
title: "Blogs"
weight: 200
bookUrlFromBaseURL: /../../blogs
relurl: "../../blogs"
menu:
main:
parent: Community
weight: 0
---
<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
Expand Down
5 changes: 0 additions & 5 deletions docs/community/_index.md

This file was deleted.

4 changes: 4 additions & 0 deletions docs/tables/configuration.md → docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@
url: configuration
aliases:
- "tables/configuration"
menu:
main:
parent: Tables
weight: 0
---
<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
Expand Down
7 changes: 5 additions & 2 deletions docs/community/contributing.md → docs/contributing.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
---
title: "Contribute"
weight: 400
bookUrlFromBaseURL: /../../contribute
relurl: "../../contributing"
menu:
main:
parent: Community
weight: 0
---
<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
Expand Down
23 changes: 0 additions & 23 deletions docs/dremio/_index.md

This file was deleted.

23 changes: 0 additions & 23 deletions docs/emr/_index.md

This file was deleted.

5 changes: 5 additions & 0 deletions docs/tables/evolution.md → docs/evolution.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
---
title: Evolution
url: evolution
aliases:
- "tables/evolution"
menu:
main:
parent: Tables
weight: 0
---
<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
Expand Down
5 changes: 5 additions & 0 deletions docs/flink/flink-connector.md → docs/flink-connector.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
---
title: "Flink Connector"
url: flink-connector
aliases:
- "flink/flink-connector"
menu:
main:
parent: Flink
weight: 200
---
<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
---
title: "Getting Started"
weight: 300
title: "Enabling Iceberg in Flink"
url: flink
menu:
main:
parent: Flink
weight: 100
---
<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
Expand Down
5 changes: 0 additions & 5 deletions docs/flink/_index.md

This file was deleted.

5 changes: 0 additions & 5 deletions docs/format/_index.md

This file was deleted.

Loading