Skip to content

Commit

Permalink
Improve documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
@ismailsimsek
ismailsimsek committed Dec 31, 2023
1 parent 32f5caf commit 0bc8358
Showing 1 changed file with 20 additions and 17 deletions.
37 changes: 20 additions & 17 deletions docs/DOCS.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,23 +15,26 @@ tables created automatically with the first start.

#### Configuration properties

| Config | Default | Description |
|----------------------------------------------------|-------------------|------------------------------------------------------------------------------------------------------------------|
| `debezium.sink.iceberg.warehouse` | | Root path of the Iceberg data warehouse |
| `debezium.sink.iceberg.catalog-name` | `default` | User-specified Iceberg catalog name. |
| `debezium.sink.iceberg.table-namespace` | `default` | A namespace in the catalog. ex: `SELECT * FROM prod.db.table -- catalog: prod, namespace: db, table: table` |
| `debezium.sink.iceberg.table-prefix` | `` | Iceberg table name prefix, Added to destination iceberg table names. |
| `debezium.sink.iceberg.write.format.default` | `parquet` | Default file format for the table; `parquet`, `avro`, or `orc` |
| `debezium.sink.iceberg.allow-field-addition` | `true` | Allow field addition to target tables |
| `debezium.sink.iceberg.upsert` | `true` | Running upsert mode overwriting updated rows. explained below. |
| `debezium.sink.iceberg.upsert-keep-deletes` | `true` | When running with upsert mode, keeps deleted rows in target table. |
| `debezium.sink.iceberg.upsert-dedup-column` | `__source_ts_ms` | With upsert mode used to deduplicate data. row with highest `__source_ts_ms` kept. _
dont change!_ |
| `debezium.sink.iceberg.upsert-op-column` | `__op` | Used with upsert mode. _dont change!_ |
| `debezium.sink.iceberg.destination-regexp` | `` | Regexp to modify destination table. With this its possible to map `table_ptt1`,`table_ptt2` to `table_combined`. |
| `debezium.sink.iceberg.destination-regexp-replace` | `` | Regexp Replace part to modify destination table |
| `debezium.sink.batch.batch-size-wait` | `NoBatchSizeWait` | Batch size wait strategy to optimize data files and upload interval. explained below. |
| `debezium.sink.iceberg.{iceberg.prop.name}` | | [Iceberg config](https://iceberg.apache.org/docs/latest/configuration/) passed to Iceberg |
| Config | Default | Description |
|------------------------------------------------------------------------------------|---------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `debezium.sink.iceberg.warehouse` | | Root path of the Iceberg data warehouse |
| `debezium.sink.iceberg.catalog-name` | `default` | User-specified Iceberg catalog name. |
| `debezium.sink.iceberg.table-namespace` | `default` | A namespace in the catalog. ex: `SELECT * FROM prod.db.table -- catalog: prod, namespace: db, table: table` |
| `debezium.sink.iceberg.table-prefix` | `` | Iceberg table name prefix, Added to destination iceberg table names. |
| `debezium.sink.iceberg.write.format.default` | `parquet` | Default file format for the table; `parquet`, `avro`, or `orc` |
| `debezium.sink.iceberg.allow-field-addition` | `true` | Allow field addition to target tables |
| `debezium.sink.iceberg.upsert` | `true` | Running upsert mode overwriting updated rows. explained below. |
| `debezium.sink.iceberg.upsert-keep-deletes` | `true` | When running with upsert mode, keeps deleted rows in target table. |
| `debezium.sink.iceberg.upsert-dedup-column` | `__source_ts_ms` | With upsert mode used to deduplicate data. row with highest `__source_ts_ms` kept. _dont change!_ |
| `debezium.sink.iceberg.upsert-op-column` | `__op` | Used with upsert mode. _dont change!_ |
| `debezium.sink.iceberg.destination-regexp` | `` | Regexp to modify destination table. With this its possible to map `table_ptt1`,`table_ptt2` to `table_combined`. |
| `debezium.sink.iceberg.destination-regexp-replace` | `` | Regexp Replace part to modify destination table |
| `debezium.sink.batch.batch-size-wait` | `NoBatchSizeWait` | Batch size wait strategy to optimize data files and upload interval. explained below. |
| `debezium.sink.iceberg.{iceberg.prop.name}` | | [Iceberg config](https://iceberg.apache.org/docs/latest/configuration/) passed to Iceberg |
| `debezium.source.offset.storage` | `io.debezium.server.iceberg.offset.IcebergOffsetBackingStore` | The name of the Java class that is responsible for persistence of connector offsets. see [debezium doc](https://debezium.io/documentation/reference/stable/development/engine.html#advanced-consuming) |
| `debezium.source.offset.storage.iceberg.table-name` | `debezium_offset_storage` | Destination table name to store connector offsets. |
| `debezium.source.schema.history.internal` | `io.debezium.server.iceberg.history.IcebergSchemaHistory` | The name of the Java class that is responsible for persistence of the database schema history. see [debezium doc](https://debezium.io/documentation/reference/stable/development/engine.html#database-history-properties) |
| `debezium.source.schema.history.internal.iceberg.table-name` | `debezium_schema_history_storage` | Destination table name to store database schema history. |

### Upsert

Expand Down

0 comments on commit 0bc8358

Please sign in to comment.