Skip to content

Commit

Permalink
[Improve][API] Add IGNORE savemode type (apache#7443)
Browse files Browse the repository at this point in the history
  • Loading branch information
jw-itq authored Aug 22, 2024
1 parent 9006a20 commit 825caa8
Show file tree
Hide file tree
Showing 13 changed files with 38 additions and 22 deletions.
3 changes: 2 additions & 1 deletion docs/en/connector-v2/sink/Doris.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,8 @@ Before the synchronous task is turned on, different treatment schemes are select
Option introduction:
`RECREATE_SCHEMA` :Will create when the table does not exist, delete and rebuild when the table is saved
`CREATE_SCHEMA_WHEN_NOT_EXIST` :Will Created when the table does not exist, skipped when the table is saved
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the table does not exist
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the table does not exist
`IGNORE` :Ignore the treatment of the table

### data_save_mode[Enum]

Expand Down
13 changes: 7 additions & 6 deletions docs/en/connector-v2/sink/Elasticsearch.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,17 +109,18 @@ Sink plugin common parameters, please refer to [Sink Common Options](../sink-com

Before the synchronous task is turned on, different treatment schemes are selected for the existing surface structure of the target side.
Option introduction:
RECREATE_SCHEMA :Will create when the table does not exist, delete and rebuild when the table is saved
CREATE_SCHEMA_WHEN_NOT_EXIST :Will Created when the table does not exist, skipped when the table is saved
ERROR_WHEN_SCHEMA_NOT_EXIST :Error will be reported when the table does not exist
`RECREATE_SCHEMA` :Will create when the table does not exist, delete and rebuild when the table is saved
`CREATE_SCHEMA_WHEN_NOT_EXIST` :Will Created when the table does not exist, skipped when the table is saved
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the table does not exist
`IGNORE` :Ignore the treatment of the table

### data_save_mode

Before the synchronous task is turned on, different processing schemes are selected for data existing data on the target side.
Option introduction:
DROP_DATA: Preserve database structure and delete data
APPEND_DATA:Preserve database structure, preserve data
ERROR_WHEN_DATA_EXISTS:When there is data, an error is reported
`DROP_DATA`: Preserve database structure and delete data
`APPEND_DATA`:Preserve database structure, preserve data
`ERROR_WHEN_DATA_EXISTS`:When there is data, an error is reported

## Examples

Expand Down
3 changes: 2 additions & 1 deletion docs/en/connector-v2/sink/Jdbc.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,8 @@ Before the synchronous task is turned on, different treatment schemes are select
Option introduction:
`RECREATE_SCHEMA` :Will create when the table does not exist, delete and rebuild when the table is saved
`CREATE_SCHEMA_WHEN_NOT_EXIST` :Will Created when the table does not exist, skipped when the table is saved
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the table does not exist
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the table does not exist
`IGNORE` :Ignore the treatment of the table

### data_save_mode [Enum]

Expand Down
1 change: 1 addition & 0 deletions docs/en/connector-v2/sink/LocalFile.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,7 @@ Existing dir processing method.
- RECREATE_SCHEMA: will create when the dir does not exist, delete and recreate when the dir is exist
- CREATE_SCHEMA_WHEN_NOT_EXIST: will create when the dir does not exist, skipped when the dir is exist
- ERROR_WHEN_SCHEMA_NOT_EXIST: error will be reported when the dir does not exist
- IGNORE :Ignore the treatment of the table

### data_save_mode [string]

Expand Down
3 changes: 2 additions & 1 deletion docs/en/connector-v2/sink/PostgreSql.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,8 @@ Before the synchronous task is turned on, different treatment schemes are select
Option introduction:
`RECREATE_SCHEMA` :Will create when the table does not exist, delete and rebuild when the table is saved
`CREATE_SCHEMA_WHEN_NOT_EXIST` :Will Created when the table does not exist, skipped when the table is saved
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the table does not exist
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the table does not exist
`IGNORE` :Ignore the treatment of the table

### data_save_mode[Enum]

Expand Down
3 changes: 2 additions & 1 deletion docs/en/connector-v2/sink/S3File.md
Original file line number Diff line number Diff line change
Expand Up @@ -281,7 +281,8 @@ Before turning on the synchronous task, do different treatment of the target pat
Option introduction:
`RECREATE_SCHEMA` :Will be created when the path does not exist. If the path already exists, delete the path and recreate it.
`CREATE_SCHEMA_WHEN_NOT_EXIST` :Will Created when the path does not exist, use the path when the path is existed.
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the path does not exist
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the path does not exist
`IGNORE` :Ignore the treatment of the table

### data_save_mode[Enum]

Expand Down
3 changes: 2 additions & 1 deletion docs/en/connector-v2/sink/StarRocks.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,8 @@ Before the synchronous task is turned on, different treatment schemes are select
Option introduction:
`RECREATE_SCHEMA` :Will create when the table does not exist, delete and rebuild when the table is saved
`CREATE_SCHEMA_WHEN_NOT_EXIST` :Will Created when the table does not exist, skipped when the table is saved
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the table does not exist
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the table does not exist
`IGNORE` :Ignore the treatment of the table

### data_save_mode[Enum]

Expand Down
15 changes: 8 additions & 7 deletions docs/zh/connector-v2/sink/Doris.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,18 +53,19 @@ Doris Sink连接器的内部实现是通过stream load批量缓存和导入的
### schema_save_mode[Enum]

在开启同步任务之前,针对现有的表结构选择不同的处理方案。
选项介绍:
选项介绍:
`RECREATE_SCHEMA` :表不存在时创建,表保存时删除并重建。
`CREATE_SCHEMA_WHEN_NOT_EXIST` :表不存在时会创建,表存在时跳过。
`ERROR_WHEN_SCHEMA_NOT_EXIST` :表不存在时会报错。
`CREATE_SCHEMA_WHEN_NOT_EXIST` :表不存在时会创建,表存在时跳过。
`ERROR_WHEN_SCHEMA_NOT_EXIST` :表不存在时会报错。
`IGNORE` :忽略对表的处理。

### data_save_mode[Enum]

在开启同步任务之前,针对目标端已有的数据选择不同的处理方案。
选项介绍:
`DROP_DATA`: 保留数据库结构并删除数据。
`APPEND_DATA`:保留数据库结构,保留数据。
`CUSTOM_PROCESSING`:用户自定义处理。
选项介绍:
`DROP_DATA`: 保留数据库结构并删除数据。
`APPEND_DATA`:保留数据库结构,保留数据。
`CUSTOM_PROCESSING`:用户自定义处理。
`ERROR_WHEN_DATA_EXISTS`:有数据时报错。

### save_mode_create_template
Expand Down
1 change: 1 addition & 0 deletions docs/zh/connector-v2/sink/Elasticsearch.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,7 @@ Sink插件常用参数,请参考 [Sink常用选项](../sink-common-options.md)
`RECREATE_SCHEMA` :当表不存在时会创建,当表已存在时会删除并重建<br/>
`CREATE_SCHEMA_WHEN_NOT_EXIST` :当表不存在时会创建,当表已存在时则跳过创建<br/>
`ERROR_WHEN_SCHEMA_NOT_EXIST` :当表不存在时将抛出错误<br/>
`IGNORE` :忽略对表的处理<br/>

### data_save_mode

Expand Down
1 change: 1 addition & 0 deletions docs/zh/connector-v2/sink/Jdbc.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,7 @@ Sink插件常用参数,请参考 [Sink常用选项](../sink-common-options.md)
`RECREATE_SCHEMA`:当表不存在时会创建,当表已存在时会删除并重建<br/>
`CREATE_SCHEMA_WHEN_NOT_EXIST`:当表不存在时会创建,当表已存在时则跳过创建<br/>
`ERROR_WHEN_SCHEMA_NOT_EXIST`:当表不存在时将抛出错误<br/>
`IGNORE` :忽略对表的处理<br/>

### data_save_mode [Enum]

Expand Down
9 changes: 5 additions & 4 deletions docs/zh/connector-v2/sink/StarRocks.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,10 +99,11 @@ table选项参数可以填入一任意表名,这个名字最终会被用作目

### schema_save_mode[Enum]

在同步任务打开之前,针对目标端已存在的表结构选择不同的处理方法。可选值有:
`RECREATE_SCHEMA` :不存在的表会直接创建,已存在的表会删除并根据参数重新创建
`CREATE_SCHEMA_WHEN_NOT_EXIST` :忽略已存在的表,不存在的表会直接创建
`ERROR_WHEN_SCHEMA_NOT_EXIST` :当有不存在的表时会直接报错
在同步任务打开之前,针对目标端已存在的表结构选择不同的处理方法。可选值有:
`RECREATE_SCHEMA` :不存在的表会直接创建,已存在的表会删除并根据参数重新创建
`CREATE_SCHEMA_WHEN_NOT_EXIST` :忽略已存在的表,不存在的表会直接创建
`ERROR_WHEN_SCHEMA_NOT_EXIST` :当有不存在的表时会直接报错
`IGNORE` :忽略对表的处理

### data_save_mode[Enum]

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,8 @@ public void handleSchemaSaveMode() {
case ERROR_WHEN_SCHEMA_NOT_EXIST:
errorWhenSchemaNotExist();
break;
case IGNORE:
break;
default:
throw new UnsupportedOperationException("Unsupported save mode: " + schemaSaveMode);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,7 @@ public enum SchemaSaveMode {

// Error will be reported when the table does not exist
ERROR_WHEN_SCHEMA_NOT_EXIST,

// Ignore creation
IGNORE
}

0 comments on commit 825caa8

Please sign in to comment.