Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update shadow api docs #12960

Merged
merged 3 commits into from
Oct 9, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions docs/document/content/dev-manual/shadow.cn.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
+++
pre = "<b>5.12. </b>"
title = "影子库"
weight = 12
chapter = true
+++

## ShadowAlgorithm

| *SPI 名称* | *详细说明* |
| ----------------------------------------- | -------------------------------- |
| ShadowAlgorithm | shadow库路由算法 |

| *已知实现类* | *详细说明* |
| ----------------------------------------- | -------------------------------- |
| ColumnRegexMatchShadowAlgorithm | 基于字段值正则匹配shadow路由算法|
| SimpleSQLNoteShadowAlgorithm | 基于SQL注解简单匹配shadow路由算法|
17 changes: 17 additions & 0 deletions docs/document/content/dev-manual/shadow.en.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
+++
pre = "<b>5.12. </b>"
title = "Shadow DB"
weight = 12
chapter = true
+++

## ShadowAlgorithm

| *SPI Name* | *Description* |
| ----------------------------------------- | ----------------------------------- |
| ShadowAlgorithm | shadow routing algorithm |

| *Implementation Class* | *Description* |
| ----------------------------------------- | ----------------------------------- |
| ColumnRegexMatchShadowAlgorithm | Column regex match shadow algorithm |
| SimpleSQLNoteShadowAlgorithm | Simple SQL note shadow algorithm |
29 changes: 25 additions & 4 deletions docs/document/content/features/shadow/concept.cn.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,35 @@ title = "核心概念"
weight = 1
+++

## 影子字段

判断该条 SQL 是否需要路由到影子数据库,为逻辑字段,数据库中不存在。

## 生产数据库

生产数据使用的数据库。

## 影子数据库

进行压测数据隔离的影子数据库,与生产数据库应当使用相同的配置。

## 影子表

进行压测数据相关的表,与生产数据库中对应表具有相同表结构。

## 影子算法

目前提供 2 种类型影子算法。
由于影子算法和业务实现紧密相关,因此并未提供默认的影子算法。

- 列影子算法

对应 ColumnShadowAlgorithm,适用于用户压测过程中,对压测执行链路上执行的SQL涉及的某个字段的值满足一定匹配条件的场景。

优点:用户只需要控制流量数据不需要修改代码和SQL就可以完成测试。

不足:仅支持DML语句

- 注解影子算法

对应 NoteShadowAlgorithm,适用于用户压测过程中,对压测执行链路上执行的SQL不能确定涉及字段的值的场景。

优点:用户可以不确定链路上执行SQL细节,只要知道那个SQL执行即可。

不足:用户需要改代码或者SQL
17 changes: 17 additions & 0 deletions docs/document/content/features/shadow/use-norms.cn.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
+++
pre = "<b>3.7.3. </b>"
title = "使用规范"
weight = 3
+++

## 支持项

* 后端数据库为 MySQL、Oracle、PostgreSQL、SQLServer;
* 支持MDL,DDL语句;
* 兼容所有常用SQL;

## 不支持项

* 影子字段无法支持范围值匹配操作,如:BETWEEN、HAVING、subQuery等;
* 使用影子库功能+分库分表功能,部分特殊SQL不支持,请参考[SQL使用规范]( https://shardingsphere.apache.org/document/current/cn/features/sharding/use-norms/sql/);
* DDL语句不支持列影子算法;
13 changes: 13 additions & 0 deletions docs/document/content/features/shadow/use-norms.en.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
+++
pre = "<b>3.7.3. </b>"
title = "Use Norms"
weight = 3
+++

## Supported Items

TODO

## Unsupported Items

TODO
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
+++
title = "影子算法"
weight = 5
+++

## 列影子算法

### 列正则表达式匹配影子算法

类型:COLUMN_REGEX_MATCH

可配置属性:

| *属性名称* | *数据类型* | *说明* |
| -------------- | --------- | ------- |
| column | String | 匹配列 |
| operation | String | SQL操作类型(insert, update, delete, select) |
| regex | String | 匹配正则表达式 |

## 注解影子算法

### 简单SQL注解匹配影子算法

类型:SIMPLE_NOTE

可配置属性:

> 至少配置一组任意的键值对。比如:foo:bar

| *属性名称* | *数据类型* | *说明* |
| -------------- | --------- | --------- |
| foo | String | bar |
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
+++
title = "Shadow Algorithm"
weight = 5
+++

## Column Shadow Algorithm

### Column Regex Match Shadow Algorithm

Type: COLUMN_REGEX_MATCH

Attributes:

| *Name* | *DataType* | *Description* |
| -------------- | --------- | ------- |
| column | String | Shadow column |
| operation | String | SQL operation type (insert, update, delete, select) |
| regex | String | Regular expression |

## Note Shadow Algorithm

### Simple SQL Note Shadow Algorithm

Type: SIMPLE_NOTE

Attributes:

> Configure at least a set of arbitrary key-value pairs. For example: foo:bar

| *Name* | *DataType* | *Description* |
| -------------- | --------- | --------- |
| foo | String | bar |
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ weight = 3
| name | String | 表名称 |
| columns (+) | Collection\<EncryptColumnRuleConfiguration\> | 加密列规则配置列表 |

### 加密列规则配置
## 加密列规则配置

类名称:org.apache.shardingsphere.encrypt.api.config.rule.EncryptColumnRuleConfiguration

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,35 @@ weight = 4

可配置属性:

| *名称* | *数据类型* | *说明* |
| -------------- | --------------------- | ----------------------------------------------------------------- |
| column | String | SQL 中的影子字段名,该值为 true 的 SQL 会路由到影子库执行 |
| sourceDataSourceNames | List\<String\> | 生产数据库名称 |
| shadowDataSourceNames | List\<String\> | 影子数据库名称,与上面一一对应 |
| *名称* | *数据类型* | *说明* | *默认值* |
| ------ | ---------- | ------ | -------- |
| enable | boolean | 影子库功能开关。可选值:true/false |false|
| dataSources | Map\<String, ShadowDataSourceConfiguration\> | 影子数据源映射名称和配置 |无|
| tables | Map\<String, ShadowTableConfiguration\> | 影子表名称和配置 |无|
| shadowAlgorithms | Map\<String, ShardingSphereAlgorithmConfiguration\> | 影子算法名称和配置 |无|

## 影子数据源配置

类名称:org.apache.shardingsphere.shadow.api.config.datasource.ShadowDataSourceConfiguration

可配置属性:

| *名称* | *数据类型* | *说明* | *默认值* |
| ------ | ---------- | ------ | -------- |
| sourceDataSourceName | String | 生产数据源名称 |无|
| shadowDataSourceName | String | 影子数据源名称 |无|

## 影子表配置

类名称:org.apache.shardingsphere.shadow.api.config.table.ShadowTableConfiguration

可配置属性:

| *名称* | *数据类型* | *说明* | *默认值* |
| ------ | ---------- | ------ | -------- |
| dataSourceNames | Collection\<String\> | 影子表关联影子数据源名称列表 |无|
| shadowAlgorithmNames | Collection\<String\> | 影子表关联影子算法名称列表 |无|

## 影子算法配置

算法类型的详情,请参见[内置影子算法列表](/cn/user-manual/shardingsphere-jdbc/configuration/built-in-algorithm/shadow)。
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,35 @@ Class name: org.apache.shardingsphere.shadow.api.config.ShadowRuleConfiguration

Attributes:

| *Name* | *DataType* | *Description* |
| --------------- | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| column | String | Shadow field name in SQL, SQL with a value of true will be routed to the shadow database for execution |
| sourceDataSourceNames | List\<String\> | Source data source names |
| shadowDataSourceNames | List\<String\> | Shadow data source names |
| *Name* | *DataType* | *Description* | *Default Value* |
| ------ | ---------- | ------------- | --------------- |
| enable | boolean | Shadow DB function switch. Optional values: true/false |false|
| dataSources | Map\<String, ShadowDataSourceConfiguration\> | Shadow data source mapping name and configuration | None |
| tables | Map\<String, ShadowTableConfiguration\> | Shadow table name and configuration | None |
| shadowAlgorithms | Map\<String, ShardingSphereAlgorithmConfiguration\> | Shadow algorithm name and configuration | None |

## Shadow Data Source Configuration

Class name: org.apache.shardingsphere.shadow.api.config.datasource.ShadowDataSourceConfiguration

Attributes:

| *Name* | *DataType* | *Description* | *Default Value* |
| ------ | ---------- | ------------- | --------------- |
| sourceDataSourceName | String | Production data source name | None |
| shadowDataSourceName | String | Shadow data source name | None |

## Shadow Table Configuration

Class name: org.apache.shardingsphere.shadow.api.config.table.ShadowTableConfiguration

Attributes:

| *Name* | *DataType* | *Description* | *Default Value* |
| ------ | ---------- | ------------- | --------------- |
| dataSourceNames | Collection\<String\> | Shadow table location shadow data source names | None |
| shadowAlgorithmNames | Collection\<String\> | Shadow table location shadow algorithm names | None |

## Shadow Algorithm Configuration

Please refer to [Built-in Shadow Algorithm List](/cn/user-manual/shardingsphere-jdbc/configuration/built-in-algorithm/shadow).
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,14 @@ weight = 4
```properties
spring.shardingsphere.datasource.names= # 省略数据源配置,请参考使用手册

spring.shardingsphere.rules.shadow.column= # 影子字段名称
spring.shardingsphere.rules.shadow.shadow-mappings.<product-data-source-name>= # 影子数据库名称
```
spring.shardingsphere.rules.shadow.enable= # Shadow功能开关。 可选值:true/false,默认为false

spring.shardingsphere.rules.shadow.data-sources.shadow-data-source.source-data-source-name= # 生产数据源名称
spring.shardingsphere.rules.shadow.data-sources.shadow-data-source.shadow-data-source-name= # 影子数据源名称

spring.shardingsphere.rules.shadow.tables.<table-name>.data-source-names= # 影子表关联影子数据源名称列表(多个值用","隔开)
spring.shardingsphere.rules.shadow.tables.<table-name>.shadow-algorithm-names= # 影子表关联影子算法名称列表(多个值用","隔开)

spring.shardingsphere.rules.shadow.shadow-algorithms.<shadow-algorithm-name>.type= # 影子算法类型
spring.shardingsphere.rules.shadow.shadow-algorithms.<shadow-algorithm-name>.props.xxx= # 影子算法属性配置
```
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,14 @@ weight = 4
```properties
spring.shardingsphere.datasource.names= # Omit the data source configuration, please refer to the usage

spring.shardingsphere.rules.shadow.column= # Shadow column name
spring.shardingsphere.rules.shadow.shadow-mappings.<product-data-source-name>= # Shadow data source name
spring.shardingsphere.rules.shadow.enable= # Shadow function switch. Optional values: true/false, the default is false

spring.shardingsphere.rules.shadow.data-sources.shadow-data-source.source-data-source-name= # Production data source name
spring.shardingsphere.rules.shadow.data-sources.shadow-data-source.shadow-data-source-name= # Shadow data source name

spring.shardingsphere.rules.shadow.tables.<table-name>.data-source-names= # Shadow table location shadow data source names (multiple values are separated by ",")
spring.shardingsphere.rules.shadow.tables.<table-name>.shadow-algorithm-names= # Shadow table location shadow algorithm names (multiple values are separated by ",")

spring.shardingsphere.rules.shadow.shadow-algorithms.<shadow-algorithm-name>.type= # Shadow algorithm type
spring.shardingsphere.rules.shadow.shadow-algorithms.<shadow-algorithm-name>.props.xxx= # Shadow algorithm property configuration
```
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,39 @@ weight = 4

\<shadow:rule />

| *名称* | *类型* | *说明* |
| ----------- | ----- | ------------------------------- |
| id | 属性 | Spring Bean Id |
| column | 属性 | 影子字段名称 |
| mappings(?) | 标签 | 生产数据库与影子数据库的映射关系配置 |
| *名称* | *类型* | *说明* |
| ----- | ------ | ------ |
| id | 属性 | Spring Bean Id |
| enable | 属性 | Shadow功能开关。 可选值:true/false,默认为false |
| data-source(?) | 标签 | 影子数据源配置 |
| shadow-table(?) | 标签 | 影子表配置 |

\<shadow:mapping />
\<shadow:data-source />

| *名称* | *类型* | *说明* |
| ------------------------ | ----- | ------------------------------- |
| product-data-source-name | 属性 | 生产数据库名称 |
| shadow-data-source-name | 属性 | 影子数据库名称 |
| *名称* | *类型* | *说明* |
| ----- | ------ | ------ |
| id | 属性 | Spring Bean Id |
| source-data-source-name | 属性 | 生产数据源名称 |
| shadow-data-source-name | 属性 | 影子数据源名称 |

\<shadow:shadow-table />

| *名称* | *类型* | *说明* |
| ----- | ------ | ------ |
| name | 属性 | 影子表名称 |
| data-sources | 属性 | 影子表关联影子数据源名称列表(多个值用","隔开)|
| algorithm (?) | 标签 | 影子表关联影子算法配置 |

\<shadow:algorithm />

| *名称* | *类型* | *说明* |
| ----- | ------ | ------ |
| shadow-algorithm-ref | 属性 | 影子表关联影子算法名称 |

\<shadow:shadow-algorithm />

| *名称* | *类型* | *说明* |
| --------- | ----- | ------------- |
| id | 属性 | 影子算法名称 |
| type | 属性 | 影子算法类型 |
| props (?) | 标签 | 影子算法属性配置 |
Loading