Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions docs/ecosystem/doris-kafka-connector.md

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

12 changes: 8 additions & 4 deletions versioned_docs/version-2.0/ecosystem/doris-kafka-connector.md

Large diffs are not rendered by default.

9 changes: 6 additions & 3 deletions versioned_docs/version-2.1/ecosystem/doris-kafka-connector.md

Large diffs are not rendered by default.

7 changes: 5 additions & 2 deletions versioned_docs/version-3.0/ecosystem/doris-kafka-connector.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,9 +197,9 @@ errors.deadletterqueue.topic.replication.factor=1
| doris.password | - | - | Y | Doris password |
| doris.database | - | - | Y | The database to write to. It can be empty when there are multiple libraries. At the same time, the specific library name needs to be configured in topic2table.map. |
| doris.topic2table.map | - | - | N | The corresponding relationship between topic and table table, for example: topic1:tb1,topic2:tb2<br />The default is empty, indicating that topic and table names correspond one to one. <br />The format of multiple libraries is topic1:db1.tbl1,topic2:db2.tbl2 |
| buffer.count.records | - | 10000 | N | The number of records each Kafka partition buffers in memory before flushing to doris. Default 10000 records |
| buffer.count.records | - | 50000 | N | The number of records each Kafka partition buffers in memory before flushing to doris. Default 50000 records |
| buffer.flush.time | - | 120 | N | Buffer refresh interval, in seconds, default 120 seconds |
| buffer.size.bytes | - | 5000000(5MB) | N | The cumulative size of records buffered in memory for each Kafka partition, in bytes, default 5MB |
| buffer.size.bytes | - | 10485760(100MB) | N | The cumulative size of records buffered in memory for each Kafka partition, in bytes, default 100MB |
| jmx | - | true | N | To obtain connector internal monitoring indicators through JMX, please refer to: [Doris-Connector-JMX](https://github.com/apache/doris-kafka-connector/blob/master/docs/en/Doris-Connector-JMX.md) |
| enable.2pc | - | true | N | Whether to enable two-phase commit (TwoPhaseCommit) of Stream Load, the default is true. |
| enable.delete | - | false | N | Whether to delete records synchronously, default false |
Expand All @@ -213,6 +213,9 @@ errors.deadletterqueue.topic.replication.factor=1
| avro.topic2schema.filepath | - | - | N | By reading the locally provided Avro Schema file, the Avro file content in the Topic is parsed to achieve decoupling from the Schema registration center provided by Confluent. <br/> This configuration needs to be used with the `key.converter` or `value.converter` prefix. For example, the local Avro Schema file for configuring avro-user and avro-product Topic is as follows: `"value.converter.avro.topic2schema. filepath":"avro-user:file:///opt/avro_user.avsc, avro-product:file:///opt/avro_product.avsc"` <br/> For specific usage, please refer to: [#32](https://github.com/apache/doris-kafka-connector/pull/32) |
| record.tablename.field | - | - | N | Configure this parameter, data from one kafka topic can flow to multiple doris tables. For configuration details, refer to: [#58](https://github.com/apache/doris-kafka-connector/pull/58) |
| enable.combine.flush | `true`,<br/> `false` | false | N | Whether to merge data from all partitions together and write them. The default value is false. When enabled, only at_least_once semantics are guaranteed.|
| max.retries | - | 10 | N | The maximum number of times to retry on errors before failing the task. |
| retry.interval.ms | - | 6000 | N | The time in milliseconds to wait following an error before attempting a retry. |
| behavior.on.null.values | `ignore`,<br/> `fail` | ignore | N | Defined how to handle records with null values. |

For other Kafka Connect Sink common configuration items, please refer to: [connect_configuring](https://kafka.apache.org/documentation/#connect_configuring)

Expand Down