Merge branch 'dev' into master

Cargo.toml

@@ -1,63 +1,7 @@
-[package]
-name = "nature"
-version = "0.22.3"
-authors = ["XueBin Li <[email protected]>"]
-edition = "2018"
-description = "It's a low code platform, it's a tool of data orchestration. But the most important is it goes right to the heart of the business, standardize and simplify the implementation of complex businesses in a simple way. As long as you're willing, Nature can help you extract the business control logic and centrally manage it so that the system has the brain and says goodbye to the brainless era of traditional systems."
-repository = "https://github.com/llxxbb/Nature"
-readme = "README.md"
-license = "MIT"
-keywords = ["platform", "data", "stream", "distributed", "management"]
-categories = ["network-programming", "database", "asynchronous", "visualization", "development-tools"]
+[workspace]
 
-[lib]
-name = "nature"           # The name of the target.
-path = "src/lib.rs"    # The source file of the target.
-
-[[bin]]
-name = "retry"
-path = "src/bin/retry.rs"
-[[bin]]
-name = "nature"
-path = "src/bin/nature.rs"
-[[bin]]
-name = "manager"
-path = "src/bin/manager.rs"
-
-[dependencies]
-# normal
-chrono = { version = "0.4", features = ["serde"] }
-serde_json = { version = "1.0", features = ["raw_value"] }
-serde = "1.0"
-serde_derive = "1.0"
-lazy_static = "1.4"
-lru_time_cache = "0.11"
-futures = "0.3"
-async-trait = "0.1"
-itertools = "0.9.0"
-uuid = { version = "0.8", features = ["v3"], optional = true }
-
-# for local executor implement
-libloading = "0.5"
-
-# log
-log = "0.4"
-env_logger = "0.7"
-
-#config
-dotenv = "0.15"
-
-# manager_lib
-reqwest = { version = "0.10", features = ["blocking", "json"] }
-actix-web = "3"
-actix-rt = "1"
-actix-cors = "0.5"
-tokio = { version = "0.2", features = ["full"] }
-
-#db
-mysql_async = "0.23"
-
-[features]
-default = ["mysql"]
-mysql = []
-sqlite = []
+members = [
+    "nature",
+    "nature-demo",
+    "test-executor"
+]

README.md

@@ -28,7 +28,7 @@ Nature 运行时模式中的 `map` 对应 `Relation`  中的 `Executor`。Nature
 
 `Executor` 在运行时会生成 `Instance`，外系统提交到 Nature 的初始数据也是 `Instance` ，`Instance` 是 `Meta` 的运行时表达，既业务的实例数据。如果您愿意您可以尽可能多地将 `Meta` 交给 Nature 来搭理，Nature 将为这些 `Meta` 所产生的 `Instance` 提供统一的、集中的存储，并为它们提供查询接口，这样 Nature 就扮演了一个数据中心的角色。这里有几点说明：
 
-- 数据检索：Nature 的业务对象都是非结构化存储的，很像 `Key-Value` 数据库。如果想对业务对象内的数据进行统计。可以利于 Nature 的流式计算机制加工出任何您想要的数据来，请参考[示例](https://github.com/llxxbb/Nature-Demo)中的销量统计。
+- 数据检索：Nature 的业务对象都是非结构化存储的，很像 `Key-Value` 数据库。如果想对业务对象内的数据进行统计。可以利于 Nature 的流式计算机制加工出任何您想要的数据来，请参考[示例](nature-demo/README.md)中的销量统计。
 - 数据库容量：Nature 缺省使用 mysql 作为后端存储，如果您的数据量很大，可以考虑使用 [Tidb](https://pingcap.com/en/) 
 
 ### 极简开发平台 
@@ -113,7 +113,7 @@ Nature 运行时模式中的 `map` 对应 `Relation`  中的 `Executor`。Nature
 
 如果您想了解下 Nature 的自然观，时空观，数学意义和哲学意义请阅读：[Nature 架构思想](doc/ZH/help/architecture.md)
 
-如果您想在实际情况中了解如何应用 Nature 请阅读：[示例及功能讲解](https://github.com/llxxbb/Nature-Demo)，[一些业务情景的解决方法](doc/ZH/help/use-case.md)
+如果您想在实际情况中了解如何应用 Nature 请阅读：[示例及功能讲解](nature-demo/README.md)，[一些业务情景的解决方法](doc/ZH/help/use-case.md)
 
 如果您想了解 Nature 的技术特性以及这些特性是如何实现的请阅读：[Nature 的技术特性](doc/ZH/help/characteristics.md)
 

README_EN.md

@@ -28,7 +28,7 @@ The `map` in Nature runtime mode corresponds to the `Executor` in `Relation`. Na
 
 `Executor` generates `Instance` at runtime. The initial data submitted to Nature by the external system is also `Instance`. `Instance` is the runtime expression of `Meta`, which is also the instance data of the business. If you want, you can hand over as much `Meta` to Nature as possible. Nature will provide unified and centralized storage for the `Instance` generated by `Meta` and provide query interfaces for them, so that Nature will played the role of a Data Center. Here are a few notes:
 
-- Data retrieval: Nature's business objects are all stored unstructured, much like the `Key-Value` database. If you want to perform statistics on the data in the business object. It can be convenient to utilize Nature’s Stream-Compute-Engine to process any data you want. Please refer to the demo of sales statistics in [Example](https://github.com/llxxbb/Nature-Demo).
+- Data retrieval: Nature's business objects are all stored unstructured, much like the `Key-Value` database. If you want to perform statistics on the data in the business object. It can be convenient to utilize Nature’s Stream-Compute-Engine to process any data you want. Please refer to the demo of sales statistics in [Example](nature-demo/README_EN.md).
 
 - Database capacity: Nature uses mysql as the back-end storage by default. If you have a large amount of data, you can consider using [Tidb](https://pingcap.com/en/).
 
@@ -112,7 +112,7 @@ In this mode, you can see the data flow
 
 If you want to understand Nature's view of nature, time and space, mathematical meaning and philosophical meaning, please read: [Nature architecture](doc/EN/help/architecture.md)
 
-If you want to learn how to apply Nature in actual situations read: [Sample and function explanation](https://github.com/llxxbb/Nature-Demo), [Solutions to some business scenarios](doc/EN/help/use-case.md)
+If you want to learn how to apply Nature in actual situations read: [Example and function explanation](nature-demo/README_EN.md), [Solutions to some business scenarios](doc/EN/help/use-case.md)
 
 If you want to know how Nature features technical features and how these features are implemented read: [Nature tecnology characteristics](doc/EN/help/characteristics.md)
 

doc/EN/help/characteristics.md

@@ -16,7 +16,7 @@ In order to achieve idempotence, Nature provides the following measures and sugg
 - Pre-allocated ID: Generate an ID before calling Nature. Maybe facebook's snowflake ID generator algorithm is a good choice. Use this ID as the ID of the `Instance`, so that when there is an environmental problem, using the same ID to submit data to Nature will not store multiple data. If you do not provide an ID, Nature will use a hash algorithm to generate one for you.
 - Para: para is useful when importing external data into Nature. At this time, Para can be the external data's ID that uniquely identifies the external data. If para is used, the ID of `Instance` is set to 0 in general.
 
-Nature supports state data, so how does Nature ensure that the data is not modified? The answer is version. Nature generates a new `Instance` for each change of the state data, but these `Instance`s in different states, they have the same ID and `Meta`, but the version number is different. Such as the order status in [Demo](https://github.com/llxxbb/Nature-Demo).
+Nature supports state data, so how does Nature ensure that the data is not modified? The answer is version. Nature generates a new `Instance` for each change of the state data, but these `Instance`s in different states, they have the same ID and `Meta`, but the version number is different. Such as the order status in [Demo](../../../nature-demo/README_EN.md).
 
 ## Rewriting protection
 
@@ -100,7 +100,7 @@ Loop -> downstream
 
 **Note**: For `MetaType::Loop`, if `MetaSetting.only_one` is set to true, Nature will treat the Instance to be output as stateful. Only in this way can the result be superimposed and can achieve input + old = new. But you cannot set the target Meta of `MetaType::Loop` as stateful! Because from the outside of Nature, we only need a final result instead of an intermediate result. It would be very strange if set to state data. In order to achieve this effect, Nature will use the intermediate result as last_state data and take it to the next batch for processing until it is completed.
 
-The batch control comes from a [built-in Executor](built-in.md) of Nature: `instance-loader`. There are some examples behind, please refer to: [demo](https://github.com/llxxbb /Nature-Demo).
+The batch control comes from a [built-in Executor](built-in.md) of Nature: `instance-loader`. There are some examples behind, please refer to: [demo](../../../nature-demo/README_EN.md).
 
 ## Context
 
@@ -112,7 +112,7 @@ The context is divided into `system context` and `user context`. User context ca
 - loop.task: used to transfer the rules between batch data, only the first batch can get the processing rules.
 - loop.finished: mark whether all batches are processed.
 
-In addition to these, there are system contexts for bridging: `target.id` and `target.para`. When there is a link A->B->C, C wants to use A's ID as its own ID, but B does not use A's ID, then B needs to build a bridge. This problem occurs when B is the data of another system. Please refer to: [Demo](https://github.com/llxxbb/Nature-Demo).
+In addition to these, there are system contexts for bridging: `target.id` and `target.para`. When there is a link A->B->C, C wants to use A's ID as its own ID, but B does not use A's ID, then B needs to build a bridge. This problem occurs when B is the data of another system. Please refer to: [Demo](../../../nature-demo/README_EN.md).
 
 There is also a system context for dynamic parameter substitution: `para.dynamic`. Generally, when configuring Relation data, we always define fixed content. But sometimes we need to determine some parameters at runtime, this time we need the context.
 

doc/EN/help/executor.md

@@ -6,7 +6,7 @@ Except for [built-in Executor](built-in.md) and automated `Executor`, you need t
 - convert_before: Used for preprocessing before the `Instance` conversion, such as data format modification, data loading, etc.
 - convert_after: Used for post-processing the `Instance`s after conversion.
 
-The three forms of `Executor` can all be found in the [Nature-Demo](https://github.com/llxxbb/Nature-Demo) project.
+The three forms of `Executor` can all be found in the [Nature-Demo](../../../nature-demo/README_EN.md) project.
 
 In fact, functionally speaking, `convert_before` and `convert_after` can be replaced by `converter` form, but Nature does not recommend, for the following reasons:
 

doc/EN/help/meta.md

@@ -82,7 +82,7 @@ The setting information of `Meta` is in JSON format, which is defined as follows
 
 - multi_meta: is a `Meta-String` array. The `Meta` whose `MetaType` is M can allow `Executor` to return multiple `Instance` of different `Meta`. The `Meta` used for returning must be defined here, and must be defined as an independent `Meta`. **Note**: `multi_meta` cannot contain the state `Meta`. **Note**: If `multi_meta` has only one value (Generally common in `Meta` whose MetaType is L), `Executor` does not need to specify the `meta` attribute of the output `Instance`, Nature will automatically fill it; if `multi_meta` is more than one Value, the output of `Executor` must clearly give the value of `Instance.meta`.
 
-- cache_saved: If true, the generated `Instance` will be cached for a short period time to avoid repeated writing to the database to improve efficiency. Common in situations where different upstream generate the same downstream, for example the `Instance` generated for time based statistical task in [Example](https://github.com/llxxbb/Nature-Demo). **Danger reminder**: Use this option incorrectly may consume a lot of memory or even overflow! You can set the `CACHE_SAVED_TIME` option in the `.env` file to change the cache time.
+- cache_saved: If true, the generated `Instance` will be cached for a short period time to avoid repeated writing to the database to improve efficiency. Common in situations where different upstream generate the same downstream, for example the `Instance` generated for time based statistical task in [Example](../../../nature-demo/README_EN.md). **Danger reminder**: Use this option incorrectly may consume a lot of memory or even overflow! You can set the `CACHE_SAVED_TIME` option in the `.env` file to change the cache time.
 - only_one: Only valid for `Meta` whose `MetaType` is L, and is used to mark whether the Loop has only one downstream `Instance` output. If it is false, each call of Loop can generate multiple `Instance` of different Meta, and these Meta given by the `multi_meta` attribute. If true, Nature regards the currently defined `Meta` as a state `Meta`, which is used to store state data each time when Loop called (the content is the `Instance` of the `Meta` specified by `multi_meta`) to serve Next time Loop, note that in this case, `multi_meta` can only define one element. The reason for processing in this way because:
 
   - `multi_meta` cannot accept state data, because processing multiple state data at the same time is extremely complex for architecture support.

doc/EN/help/relation.md

@@ -178,4 +178,4 @@ After the execution of `Executor` is completed, sometimes we want to append some
 {"para.dynamic":"[[\"key\",\"value\"]]"}
 ```
 
-The key derived from the value corresponding to `dynamic_para`, and the value derived from the additional value generated by `append_para`. The function of `para.dynamic` is to replace the variables in `Executor.settings`, please refer to the sales statistics in [Demo](https://github.com/llxxbb/Nature-Demo).
+The key derived from the value corresponding to `dynamic_para`, and the value derived from the additional value generated by `append_para`. The function of `para.dynamic` is to replace the variables in `Executor.settings`, please refer to the sales statistics in [Demo](../../../nature-demo/README_EN.md).

doc/ZH/help/architecture.md

@@ -134,7 +134,7 @@ Nature 是适合快速迭代的，它不存在系统间的**边界墙**问题，
 
 ### 选择所实现的无形控制
 
-在 [Demo](https://github.com/llxxbb/Nature-Demo) 中 涉及到网购和统计相关的示例，这些示例说明了 Nature 如何简化这些业务的实现。在这里不做具体展开，这里只想说明一下选择机制如何有效支撑系统的运行秩序。为了简单起见，所表达的内容可能与Demo中的不完全一致，还请谅解。
+在 [Demo](../../../nature-demo/README.md) 中 涉及到网购和统计相关的示例，这些示例说明了 Nature 如何简化这些业务的实现。在这里不做具体展开，这里只想说明一下选择机制如何有效支撑系统的运行秩序。为了简单起见，所表达的内容可能与Demo中的不完全一致，还请谅解。
 
 上文中我们说到选择是下游对上游的选择，这就揭示了一种思考方式：**逆向思维**，既我们要达到目的需要什么。拿网购来讲，需要从流程的终点来倒推。用户若想拿到商品需要配送员送，交接数据为签收单，于是我们定义第一个`Meta`。然后我们再倒推，配送员需要和库房交接出库单才能拿到商品进行配送，出库单是我们的第二个`Meta`于是我们有了第一个`Relation` ： 出库单->签收单。
 

doc/ZH/help/characteristics.md

@@ -16,7 +16,7 @@ Nature 只能插入数据不能变更数据，`Instance`一旦生成既被永久
 - 预分配ID：在调用Nature 之前预先生成一个ID，或许 facebook 的 snowflake ID 生成器算法是一个不错的选择。使用此ID作为`Instance`的ID，这样当出现环境问题时使用相同的ID提交数据到 Nature 就不会存储多条数据了。如果你不提供ID，Nature 会使用哈希算法为你生成一个。
 - para: 外部已经存在的数据导入 Nature 时 para 会很有用。此时 Para 可以是唯一标识外部数据的ID。如果使用 para，一般情况下 `Instance` 的 ID 置0。
 
-Nature 是支持状态数据的，那么 Nature 如何保证数据不被修改？答案是版本。Nature 为状态数据的每一次状态变更都会生成一个新的`Instance`，但这些不同状态的`Instance`拥有相同的ID和`Meta`，只是版本号是不同的。如 [Demo](https://github.com/llxxbb/Nature-Demo) 中的订单状态。
+Nature 是支持状态数据的，那么 Nature 如何保证数据不被修改？答案是版本。Nature 为状态数据的每一次状态变更都会生成一个新的`Instance`，但这些不同状态的`Instance`拥有相同的ID和`Meta`，只是版本号是不同的。如 [Demo](../../../nature-demo/README.md) 中的订单状态。
 
 ## 防重机制
 
@@ -99,7 +99,7 @@ Loop -> downstream
 
 **注意**：对于 `MetaType::Loop` 来讲 `MetaSetting.only_one`如果设置为 true, Nature 会将要输出的 Instance 视为有状态的，只有这样才能实现结果的叠加，才能完成形如 input + old = new 这种形式的数据处理。但你不能把`MetaType::Loop` 的目标 Meta 设置为有状态的！因为从 Nature 外部来看我们只要一个最终结果而不是中间结果，如果置为状态数据会让人感觉到非常奇怪。为了实现这种效果，Nature会把中间结果作为 last_state 数据并带到下一个批次里处理直到完成为止。
 
-批量的控制来源于 Nature 的一个[内置Executor](built-in.md)：`instance-loader` 后面有这样的示例，请参考：[示例及功能讲解](https://github.com/llxxbb/Nature-Demo)。
+批量的控制来源于 Nature 的一个[内置Executor](built-in.md)：`instance-loader` 后面有这样的示例，请参考：[示例及功能讲解](../../../nature-demo/README.md)。
 
 ## 上下文
 
@@ -111,7 +111,7 @@ Loop -> downstream
 - loop.task：用于传递批数据的处理规则，只有第一个批次可以取得处理规则。
 - loop.finished：标记所有批次是否处理完成。
 
-除了这些外，还有用于桥接的系统上下文：`target.id` 和 `target.para`。当有 A->B->C的链路时，C想使用A的ID作为自己的ID,而B没有使用A的ID，这时候就需要B架一个桥了。当B为另一个体系的数据时会有这个问题。请参考：[示例及功能讲解](https://github.com/llxxbb/Nature-Demo)。
+除了这些外，还有用于桥接的系统上下文：`target.id` 和 `target.para`。当有 A->B->C的链路时，C想使用A的ID作为自己的ID,而B没有使用A的ID，这时候就需要B架一个桥了。当B为另一个体系的数据时会有这个问题。请参考：[示例及功能讲解](../../../nature-demo/README.md)。
 
 还有用于动态参数替换的系统上下文：`para.dynamic`。一般我们在配置 Relation 数据时，都是定义好的固定内容。但有时候我们需要运行时确定一些参数，这时候就需要该上下文了。
 

doc/ZH/help/Executor.md → doc/ZH/help/executor.md

@@ -6,7 +6,7 @@
 - convert_before: 用于 `Instance` 转换前的预处理，如数据格式的修正，数据加载等。
 - convert_after: 用于转换后 `Instance` 的后置处理。
 
-这三种形式的 `Executor` 都可以在 [Nature-Demo](https://github.com/llxxbb/Nature-Demo) 项目中找到对应的示例。
+这三种形式的 `Executor` 都可以在 [Nature-Demo](../../../nature-demo/README.md) 项目中找到对应的示例。
 
 其实从功能上讲 `convert_before` 和 `convert_after` 完全可以用 `converter` 形式来替换，但 Nature 不建议这样做，有下面的原因：