doc: add extension standalone testing framework (#71)

docs/SUMMARY.md

@@ -24,13 +24,17 @@
 * [Type System](ten_framework/type_system.md)
 * [Schema System](ten_framework/schema_system.md)
 * [Build System](ten_framework/build_system.md)
-* [Test System](ten_framework/test_system.md)
 * [Graph](ten_framework/graph.md)
 * [Cloud Store](ten_framework/cloud_store.md)
 * [Debugging](ten_framework/debugging.md)
 * [Profiling](ten_framework/profiling.md)
 * [Dependencies](ten_framework/dependencies.md)
 
+### TEN Testing System
+
+* [For Users of TEN framework](ten_framework/ten_testing/for_users_of_ten_framework.md)
+* [For Developers of TEN framework](ten_framework/ten_testing/for_developers_of_ten_framework.md)
+
 ### Binding
 
 * [Go](ten_framework/binding/go.md)

docs/ten_framework/test_system.md → docs/ten_framework/ten_testing/for_developers_of_ten_framework.md

@@ -1,10 +1,4 @@
-# Test System
-
-## For Users of the TEN Framework
-
-### Standalone Testing of Extension
-
-## For Developers of the TEN Framework
+# For Developers of the TEN Framework
 
 The TEN framework includes three types of test suites:
 
@@ -14,7 +8,7 @@ The TEN framework includes three types of test suites:
 
 The TEN framework uses `gtest` as the testing framework for unit tests and smoke tests, and `pytest` as the testing framework for integration tests.
 
-### Unit Tests
+## Unit Tests
 
 The source directory for unit tests is `tests/ten_runtime/unit`. If you need to add new unit test cases, please place them in the `tests/ten_runtime/unit directory`.
 
@@ -24,7 +18,7 @@ For C++ projects, after successful compilation, you can navigate to the output d
 ./ten_runtime_unit_test
 ```
 
-### Smoke Tests
+## Smoke Tests
 
 The source directory for smoke tests is `tests/ten_runtime/smoke`. If you need to add new smoke test cases, please place them in the `tests/ten_runtime/smoke` directory.
 
@@ -34,7 +28,7 @@ Navigate to the output directory (e.g., `out/linux/x64/tests/standalone`, depend
 ./ten_runtime_smoke_test
 ```
 
-### Loop Testing
+## Loop Testing
 
 You can perform multiple rounds of testing using the following commands.
 
@@ -56,7 +50,7 @@ To run both unit tests and smoke tests:
 failed=0; for i in {1..100}; do if [ ${failed} != 0 ]; then echo "error occurred:" ${failed}; break; fi; ./ten_runtime_unit_test; failed=$?; if [ ${failed} != 0 ]; then echo "error occurred:" ${failed}; break; fi; ./ten_runtime_smoke_test; failed=$?; done;
 ```
 
-### Integration Tests
+## Integration Tests
 
 Integration tests are black-box tests for the TEN framework. The source directory for integration tests is `tests/ten_runtime/integration`. These tests are used to validate real-world execution scenarios.
 

docs/ten_framework/ten_testing/for_users_of_ten_framework.md

@@ -0,0 +1,109 @@
+# For Users of the TEN Framework
+
+## Standalone Testing of Extension
+
+The TEN framework provides an independent extension testing mechanism. This mechanism allows developers to test extensions without relying on other TEN concepts, such as other TEN extensions, TEN graphs, or TEN apps. When developers need to test the behavior of an extension without running the entire TEN app, this independent extension testing mechanism is especially useful.
+
+The standalone testing framework for TEN extensions follows two key principles:
+
+1. Compatibility with any native testing framework of the used language.
+
+   For example, if a TEN extension is developed using C++, the Google gtest/gmock testing framework can be used alongside the TEN extension standalone testing framework to achieve independent testing of the C++ TEN extension.
+
+2. Users don't need to modify any code in the extension under test.
+
+   The exact same extension code used in runtime can be fully tested using this standalone testing framework.
+
+For users, the TEN extension standalone testing framework introduces two main concepts:
+
+1. **extension_tester**
+2. **ten_env_tester**
+
+The role of **extension_tester** is similar to a testing driver, responsible for setting up and executing the entire testing process. **ten_env_tester**, on the other hand, behaves like a typical TEN extension's `ten_env` instance, enabling users to invoke functionalities within the standalone testing framework from the callback of `extension_tester`, such as sending messages to the extension under test and receiving messages from the extension.
+
+## Standalone Testing Framework Internal
+
+Internally, the TEN extension standalone testing framework implicitly starts a test app, loads the extension addon to be tested (let’s call it extension A), and initializes a graph containing both the extension A to be tested and another extension (let’s call it extension B) dedicated to testing other extensions. All input and output messages of extension A are redirected to extension B, allowing users to customize inputs and outputs during the testing process and complete the actual testing work.
+
+```mermaid
+sequenceDiagram
+  participant tester as Extension Tester
+  participant testing as Extension Testing
+  participant tested as Extension Tested
+
+  tester->>testing
+  testing->>tested
+
+  tested-->>testing
+  testing-->>tester
+```
+
+## C++
+
+Here is an example of TEN extension standalone testing using Google gtest:
+
+```c++
+class extension_tester_basic : public ten::extension_tester_t {
+ public:
+  void on_start(ten::ten_env_tester_t &ten_env) override {
+    auto new_cmd = ten::cmd_t::create("hello_world");
+    ten_env.send_cmd(std::move(new_cmd),
+                     [](ten::ten_env_tester_t &ten_env,
+                        std::unique_ptr<ten::cmd_result_t> result) {
+                       if (result->get_status_code() == TEN_STATUS_CODE_OK) {
+                         ten_env.stop_test();
+                       }
+                     });
+  }
+};
+
+TEST(Test, Basic) {
+  auto *tester = new extension_tester_basic();
+
+  ten_string_t *path = ten_path_get_executable_path();
+  ten_path_join_c_str(path, "../ten_packages/extension/default_extension_cpp/");
+
+  tester->add_addon_base_dir(ten_string_get_raw_str(path));
+
+  ten_string_destroy(path);
+
+  tester->set_test_mode_single("default_extension_cpp");
+  tester->run();
+  delete tester;
+}
+```
+
+## Golang
+
+TODO: To be added.
+
+## Python
+
+```python
+class ExtensionTesterBasic(ExtensionTester):
+    def check_hello(self, ten_env: TenEnvTester, result: CmdResult):
+        statusCode = result.get_status_code()
+        print("receive hello_world, status:" + str(statusCode))
+
+        if statusCode == StatusCode.OK:
+            ten_env.stop_test()
+
+    def on_start(self, ten_env: TenEnvTester) -> None:
+        new_cmd = Cmd.create("hello_world")
+
+        print("send hello_world")
+        ten_env.send_cmd(
+            new_cmd,
+            lambda ten_env, result: self.check_hello(ten_env, result),
+        )
+
+        print("tester on_start_done")
+        ten_env.on_start_done()
+
+
+def test_basic():
+    tester = ExtensionTesterBasic()
+    tester.add_addon_base_dir(str(Path(__file__).resolve().parent.parent))
+    tester.set_test_mode_single("default_extension_python")
+    tester.run()
+```