From ec184c19fb264698747c378dd87774039ec1f9bd Mon Sep 17 00:00:00 2001 From: Hu Yueh-Wei Date: Thu, 29 Aug 2024 16:08:54 +0800 Subject: [PATCH] fix --- .../how-to-build-extension-with-c++-beta.md | 1118 ---------------- .../how-to-build-extension-with-go-beta.md | 1120 ----------------- tutorials/how-to-debug-with-logs.md | 47 - 3 files changed, 2285 deletions(-) delete mode 100644 tutorials/how-to-build-extension-with-c++-beta.md delete mode 100644 tutorials/how-to-build-extension-with-go-beta.md delete mode 100644 tutorials/how-to-debug-with-logs.md diff --git a/tutorials/how-to-build-extension-with-c++-beta.md b/tutorials/how-to-build-extension-with-c++-beta.md deleted file mode 100644 index dc85eb9..0000000 --- a/tutorials/how-to-build-extension-with-c++-beta.md +++ /dev/null @@ -1,1118 +0,0 @@ ---- -hidden: true -layout: - title: - visible: true - description: - visible: false - tableOfContents: - visible: true - outline: - visible: true - pagination: - visible: true ---- - -# 🚧 How to build extension with C++(beta) - -## Overview - -This tutorial introduces how to develop an TEN extension using C++, as well as how to debug and deploy it to run in an TEN app. This tutorial covers the following topics: - -* How to create a C++ extension development project using arpm. -* How to use TEN API to implement the functionality of the extension, such as sending and receiving messages. -* How to write unit test cases and debug the code. -* How to deploy the extension locally to an app and perform integration testing within the app. -* How to debug the extension code within the app. - -Note - -Unless otherwise specified, the commands and code in this tutorial are executed in a Linux environment. Since TEN has a consistent development approach and logic across all platforms (e.g., Windows, Mac), this tutorial is also suitable for other platforms. - -### Preparation - -* Download the latest arpm and configure the PATH. You can check if it is configured correctly with the following command: - - ``` - $ arpm -h - ``` - - If the configuration is successful, it will display the help information for arpm as follows: - - ``` - Usage: arpm [OPTIONS] - - Commands: - install Install a package. For more detailed usage, run 'install -h' - publish Publish a package. For more detailed usage, run 'publish -h' - dev-server Install a package. For more detailed usage, run 'dev-server -h' - help Print this message or the help of the given subcommand(s) - - Options: - --config-file The location of config.json - -h, --help Print help - -V, --version Print version - ``` -* Download the latest standalone\_gn and configure the PATH. For example: - - Note - - standalone\_gn is the C++ build system for the TEN platform. To facilitate developers, TEN provides a standalone\_gn toolchain for building C++ extension projects. - - ``` - $ export PATH=/path/to/standalone_gn:$PATH - ``` - - You can check if the configuration is successful with the following command: - - ``` - $ ag -h - ``` - - If the configuration is successful, it will display the help information for standalone\_gn as follows: - - ``` - usage: ag [-h] [-v] [--verbose | --no-verbose] [--out_file OUT_FILE] [--out_dir OUT_DIR] command target_OS target_CPU build_type - - An easy-to-use Google gn wrapper - - positional arguments: - command possible commands are: - gen build rebuild refs clean - graph uninstall explain_build desc check - show_deps show_input show_input_output path args - target_OS possible OS values are: - win mac ios android linux - target_CPU possible values are: - x64 x64 arm arm64 - build_type possible values are: - debug release - - options: - -h, --help show this help message and exit - -v, --version show program's version number and exit - --verbose, --no-verbose - dump verbose outputs - --out_file OUT_FILE dump command output to a file - --out_dir OUT_DIR build output dir, default is 'out/' - ``` - - Note - - * gn depends on python3, please make sure that Python 3.10 or above is installed. -* Install a C/C++ compiler, either clang/clang++ or gcc/g++. - -In addition, we provide a base compilation image where all of the above dependencies are already installed and configured. You can refer to the [ASTRA.ai](https://github.com/rte-design/ASTRA.ai) project on GitHub. - -### Creating C++ extension project - - - -#### Creating Based on Templates - - - -Assuming we want to create a project named first\_cxx\_extension, we can use the following command to create it: - -``` -$ arpm install extension default_extension_cpp --template-mode --template-data package_name=first_cxx_extension -``` - -Note - -The above command indicates that we are installing an TEN package using the default\_extension\_cpp template to create an extension project named first\_cxx\_extension. - -* \--template-mode indicates installing the TEN package as a template. The template rendering parameters can be specified using --template-data. -* extension is the type of TEN package to install. Currently, TEN provides app/extension\_group/extension/system packages. In the following sections on testing extensions in an app, we will use several other types of packages. -* default\_extension\_cpp is the default C++ extension provided by TEN. Developers can also specify other C++ extensions available in the store as templates. - -After the command is executed, a directory named first\_cxx\_extension will be generated in the current directory, which is our C++ extension project. The directory structure is as follows: - -``` -. -β”œβ”€β”€ BUILD.gn -β”œβ”€β”€ manifest.json -β”œβ”€β”€ property.json -└── src - └── main.cc -``` - -Where: - -* src/main.cc contains a simple implementation of the extension, including calls to the C++ API provided by TEN. We will discuss how to use the TEN API in the next section. -* manifest.json and property.json are the standard configuration files for TEN extensions. In manifest.json, metadata information such as the version, dependencies, and schema definition of the extension are typically declared. property.json is used to declare the business configuration of the extension. -* BUILD.gn is the configuration file for standalone\_gn, used to compile the C++ extension project. - -The property.json file is initially an empty JSON file, like this: - -``` -{} -``` - -The manifest.json file will include the rte\_runtime dependency by default, like this: - -``` -{ - "type": "extension", - "name": "first_cxx_extension", - "version": "0.2.0", - "language": "cpp", - "dependencies": [ - { - "type": "system", - "name": "rte_runtime", - "version": "0.2.0" - } - ], - "api": {} -} -``` - -Note - -* Please note that according to TEN's naming convention, the name should be alphanumeric. This is because when integrating the extension into an app, a directory will be created based on the extension name. TEN also provides the functionality to automatically load the manifest.json and property.json files from the extension directory. -* Dependencies are used to declare the dependencies of the extension. When installing TEN packages, arpm will automatically download the dependencies based on the declarations in the dependencies section. -* The api section is used to declare the schema of the extension. Refer to `usage of rte schema `. - -#### Manual Creation - - - -Developers can also manually create a C++ extension project or transform an existing project into an TEN extension project. - -First, ensure that the project's output target is a shared library. Then, refer to the example above to create `property.json` and `manifest.json` in the project's root directory. The `manifest.json` should include information such as `type`, `name`, `version`, `language`, and `dependencies`. Specifically: - -* `type` must be `extension`. -* `language` must be `cpp`. -* `dependencies` should include `rte_runtime`. - -Finally, configure the build settings. The `default_extension_cpp` provided by TEN uses `standalone_gn` as the build toolchain. If developers are using a different build toolchain, they can refer to the configuration in `BUILD.gn` to set the compilation parameters. Since `BUILD.gn` contains the directory structure of the TEN package, we will discuss it in the next section (Downloading Dependencies). - -### Download Dependencies - - - -To download dependencies, execute the following command in the extension project directory: - -``` -$ arpm install -``` - -After the command is executed successfully, a `.rte` directory will be generated in the current directory, which contains all the dependencies of the current extension. - -Note - -* There are two modes for extensions: development mode and runtime mode. In development mode, the root directory is the source code directory of the extension. In runtime mode, the root directory is the app directory. Therefore, the placement path of dependencies is different in these two modes. The `.rte` directory mentioned here is the root directory of dependencies in development mode. - -The directory structure is as follows: - -``` -. -β”œβ”€β”€ BUILD.gn -β”œβ”€β”€ manifest.json -β”œβ”€β”€ property.json -β”œβ”€β”€ .rte -β”‚ └── app -β”‚ β”œβ”€β”€ addon -β”‚ β”œβ”€β”€ include -β”‚ └── lib -└── src - └── main.cc -``` - -Where: - -* `.rte/app/include` is the root directory for header files. -* `.rte/app/lib` is the root directory for precompiled dynamic libraries of TEN runtime. - -If it is in runtime mode, the extension will be placed in the `addon/extension` directory of the app, and the dynamic libraries will be placed in the `lib` directory of the app. The structure is as follows: - -``` -. -β”œβ”€β”€ BUILD.gn -β”œβ”€β”€ manifest.json -β”œβ”€β”€ property.json -β”œβ”€β”€ addon -β”‚ └── extension -β”‚ └── first_cxx_extension -β”œβ”€β”€ include -└── lib -``` - -So far, an TEN C++ extension project has been created. - -### BUILD.gn - - - -The content of `BUILD.gn` for `default_extension_cpp` is as follows: - -``` -import("//exts/rte/base_options.gni") -import("//exts/rte/rte_package.gni") - -config("common_config") { - defines = common_defines - include_dirs = common_includes - cflags = common_cflags - cflags_c = common_cflags_c - cflags_cc = common_cflags_cc - cflags_objc = common_cflags_objc - cflags_objcc = common_cflags_objcc - libs = common_libs - lib_dirs = common_lib_dirs - ldflags = common_ldflags -} - -config("build_config") { - configs = [ ":common_config" ] - - # 1. The `include` refers to the `include` directory in current extension. - # 2. The `//include` refers to the `include` directory in the base directory - # of running `ag gen`. - # 3. The `.rte/app/include` is used in extension standalone building. - include_dirs = [ - "include", - "//include", - "//include/nlohmann_json", - ".rte/app/include", - ".rte/app/include/nlohmann_json", - ] - - lib_dirs = [ - "lib", - "//lib", - ".rte/app/lib", - ] - - if (is_win) { - libs = [ - "rte_runtime.dll.lib", - "utils.dll.lib", - ] - } else { - libs = [ - "rte_runtime", - "utils", - ] - } -} - -rte_package("first_cxx_extension") { - package_type = "develop" # develop | release - package_kind = "extension" - - manifest = "manifest.json" - property = "property.json" - - if (package_type == "develop") { - # It's 'develop' package, therefore, need to build the result. - build_type = "shared_library" - - sources = [ "src/main.cc" ] - - configs = [ ":build_config" ] - } -} -``` - -Let's first take a look at the `rte_package` target, which declares a build target for an TEN package. - -* The `package_kind` is set to `extension`, and the `build_type` is set to `shared_library`. This means that the expected output of the compilation is a shared library. -* The `sources` field specifies the source file(s) to be compiled. If there are multiple source files, they need to be added to the `sources` field. -* The `configs` field specifies the build configurations. It references the `build_config` defined in this file. - -Next, let's look at the content of `build_config`. - -* The `include_dirs` field defines the search paths for header files. - * The difference between `include` and `//include` is that `include` refers to the `include` directory in the current extension directory, while `//include` is based on the working directory of the `ag gen` command. So, if the compilation is executed in the extension directory, it will be the same as `include`. But if it is executed in the app directory, it will be the `include` directory in the app. - * `.rte/app/include` is used for standalone development and compilation of the extension, which is the scenario being discussed in this tutorial. In other words, the default `build_config` is compatible with both development mode and runtime mode compilation. -* The `lib_dirs` field defines the search paths for dependency libraries. The difference between `lib` and `//lib` is similar to `include`. -* The `libs` field defines the dependent libraries. `rte_runtime` and `utils` are libraries provided by TEN. - -Therefore, if developers are using a different build toolchain, they can refer to the above configuration and set the compilation parameters in their own build toolchain. For example, if using g++ to compile: - -``` -$ g++ -shared -fPIC -I.rte/app/include/ -L.rte/app/lib -lrte_runtime -lutils -Wl,-rpath=\$ORIGIN -Wl,-rpath=\$ORIGIN/../../../lib src/main.cc -``` - -The setting of `rpath` is also considered for the runtime mode, where the rte\_runtime dependency of the extension is placed in the `app/lib` directory. - -### Implementation of Extension Functionality - - - -For developers, there are two things to do: - -* Create an extension as a channel for interacting with TEN runtime. -* Register the extension as an addon in TEN, allowing it to be used in the graph through a declarative approach. - -#### Creating the Extension Class - - - -The extension created by developers needs to inherit the `rte::extension_t` class. The main definition of this class is as follows: - -``` -class extension_t { -protected: - explicit extension_t(const std::string &name) {...} - - virtual void on_init(rte_t &rte, metadata_info_t &manifest, - metadata_info_t &property) { - rte.on_init_done(manifest, property); - } - - virtual void on_start(rte_t &rte) { rte.on_start_done(); } - - virtual void on_stop(rte_t &rte) { rte.on_stop_done(); } - - virtual void on_deinit(rte_t &rte) { rte.on_deinit_done(); } - - virtual void on_cmd(rte_t &rte, std::unique_ptr cmd) { - auto cmd_result = rte::cmd_result_t::create(RTE_STATUS_CODE_OK); - cmd_result->set_property("detail", "default"); - rte.return_result(std::move(cmd_result), std::move(cmd)); - } - - virtual void on_data(rte_t &rte, std::unique_ptr data) {} - - virtual void on_pcm_frame(rte_t &rte, std::unique_ptr frame) {} - - virtual void on_image_frame(rte_t &rte, - std::unique_ptr frame) {} -} -``` - -In the markdown content you provided, there are descriptions of the lifecycle functions and message handling functions in Chinese. Here is the translation: - -Lifecycle Functions: - -* on\_init: Used to initialize the extension instance, such as setting the extension's configuration. -* on\_start: Used to start the extension instance, such as establishing connections to external services. The extension will not receive messages until on\_start is completed. In on\_start, you can use the rte.get\_property API to retrieve the extension's configuration. -* on\_stop: Used to stop the extension instance, such as closing connections to external services. -* on\_deinit: Used to destroy the extension instance, such as releasing memory resources. - -Message Handling Functions: - -* on\_cmd/on\_data/on\_pcm\_frame/on\_image\_frame: These are callback methods used to receive messages of four different types. For more information on TEN message types, you can refer to the [message-type-and-name](https://github.com/rte-design/ASTRA.ai/blob/main/docs/message-type-and-name.md) - -The rte::extension\_t class provides default implementations for these functions, and developers can override them according to their needs. - -#### Registering the Extension - - - -After defining the extension, it needs to be registered as an addon in the TEN runtime. For example, in the `first_cxx_extension/src/main.cc` file, the registration code is as follows: - -``` -RTE_CPP_REGISTER_ADDON_AS_EXTENSION(first_cxx_extension, first_cxx_extension_extension_t); -``` - -* RTE\_CPP\_REGISTER\_ADDON\_AS\_EXTENSION is a macro provided by the TEN runtime for registering extension addons. - * The first parameter is the name of the addon, which serves as a unique identifier for the addon. It will be used to define the extension in the graph using a declarative approach. - * The second parameter is the implementation class of the extension, which is the class that inherits from rte::extension\_t. - -Please note that the addon name must be unique because it is used as a unique index to find the implementation in the graph. - -#### on\_init - - - -Developers can set the extension's configuration in the on\_init() function, as shown in the example: - -``` -void on_init(rte::rte_t& rte, rte::metadata_info_t& manifest, - rte::metadata_info_t& property) override { - property.set(RTE_METADATA_JSON_FILENAME, "customized_property.json"); - rte.on_init_done(manifest, property); -} -``` - -Both the property and manifest can be customized using the set() method. In the example, the first parameter RTE\_METADATA\_JSON\_FILENAME indicates that the custom property is stored as a local file, and the second parameter is the file path relative to the extension directory. So in this example, when the app loads the extension, it will load `/addon/extension/first_cxx_extension/customized_property.json`. - -TEN's on\_init provides default logic for loading default configurations. If developers do not call property.set(), the property.json file in the extension directory will be loaded by default. Similarly, if manifest.set() is not called, the manifest.json file in the extension directory will be loaded by default. In the example, since property.set() is called, the property.json file will not be loaded by default. - -Please note that on\_init is an asynchronous method, and developers need to call rte.on\_init\_done() to inform the TEN runtime that on\_init has completed as expected. - -#### on\_start - - - -When on\_start is called, it means that on\_init\_done() has been executed and the extension's property has been loaded. From this point on, the extension can access the configuration. For example: - -``` -void on_start(rte::rte_t& rte) override { - auto prop = rte.get_property_string("some_string"); - // do something - - rte.on_start_done(); -} -``` - -rte.get\_property\_string() is used to retrieve a property of type string with the name "some\_string". If the property does not exist or the type does not match, an error will be returned. If the extension's configuration contains the following content: - -``` -{ - "some_string": "hello world" -} -``` - -Then the value of prop will be "hello world". - -Similar to on\_init, on\_start is also an asynchronous method, and developers need to call rte.on\_start\_done() to inform the TEN runtime that on\_start has completed as expected. - -For more information, you can refer to the API documentation: rte api doc. - -#### Error Handling - - - -As shown in the previous example, if "some\_string" does not exist or is not of type string, rte.get\_property\_string() will return an error. You can handle the error as follows: - -``` -void on_start(rte::rte_t& rte) override { - rte::error_t err; - auto prop = rte.get_property_string("some_string", &err); - - // error handling - if (!err.is_success()) { - RTE_LOGE("Failed to get property: %s", err.errmsg()); - } - - rte.on_start_done(); -} -``` - -#### Message Handling - - - -TEN provides four types of messages: `cmd`, `data`, `image_frame`, and `pcm_frame`. Developers can handle these four types of messages by implementing the `on_cmd`, `on_data`, `on_image_frame`, and `on_pcm_frame` callback methods. - -Taking `cmd` as an example, let's see how to receive and send messages. - -Assume that `first_cxx_extension` receives a `cmd` with the name `hello`, which includes the following properties: - -| name | type | -| ---------------- | ------ | -| app\_id | string | -| client\_type | int8 | -| payload | object | -| payload.err\_no | uint8 | -| payload.err\_msg | string | - -The processing logic of `first_cxx_extension` for the `hello` cmd is as follows: - -* If the `app_id` or `client_type` parameters are invalid, return an error: - - ``` - { - "err_no": 1001, - "err_msg": "Invalid argument." - } - ``` -* If `payload.err_no` is greater than 0, return an error with the content from the `payload`. -* If `payload.err_no` is equal to 0, forward the `hello` cmd downstream for further processing. After receiving the processing result from the downstream extension, return the result. - -**Describing the Extension's Behavior in manifest.json** - - - -Based on the above description, the behavior of `first_cxx_extension` is as follows: - -* It receives a `cmd` named `hello` with properties. -* It may send a `cmd` named `hello` with properties. -* It receives a response from a downstream extension, which includes error information. -* It returns a response to an upstream extension, which includes error information. - -For a TEN extension, you can describe the above behavior in the `manifest.json` file of the extension, including: - -* What messages the extension receives, their names, and the structure definition of their properties (schema definition). -* What messages the extension generates/sends, their names, and the structure definition of their properties. -* Additionally, for `cmd` type messages, a response definition is required (referred to as a result in TEN). - -With these definitions, TEN runtime will perform validity checks based on the schema definition before delivering messages to the extension or when the extension sends messages through TEN runtime. It also helps the users of the extension to see the protocol definition. - -The schema is defined in the `api` field of the `manifest.json` file. `cmd_in` defines the cmds that the extension will receive, and `cmd_out` defines the cmds that the extension will send. - -Note - -For the usage of schema, refer to: [rte-schema](https://github.com/rte-design/ASTRA.ai/blob/main/docs/rte-schema.md) - -Based on the above description, the content of `manifest.json` for `first_cxx_extension` is as follows: - -``` -{ - "type": "extension", - "name": "first_cxx_extension", - "version": "0.2.0", - "language": "cpp", - "dependencies": [ - { - "type": "system", - "name": "rte_runtime", - "version": "0.2.0" - } - ], - "api": { - "cmd_in": [ - { - "name": "hello", - "property": { - "app_id": { - "type": "string" - }, - "client_type": { - "type": "int8" - }, - "payload": { - "type": "object", - "properties": { - "err_no": { - "type": "uint8" - }, - "err_msg": { - "type": "string" - } - } - } - }, - "required": ["app_id", "client_type"], - "result": { - "property": { - "err_no": { - "type": "uint8" - }, - "err_msg": { - "type": "string" - } - }, - "required": ["err_no"] - } - } - ], - "cmd_out": [ - { - "name": "hello", - "property": { - "app_id": { - "type": "string" - }, - "client_type": { - "type": "string" - }, - "payload": { - "type": "object", - "properties": { - "err_no": { - "type": "uint8" - }, - "err_msg": { - "type": "string" - } - } - } - }, - "required": ["app_id", "client_type"], - "result": { - "property": { - "err_no": { - "type": "uint8" - }, - "err_msg": { - "type": "string" - } - }, - "required": ["err_no"] - } - } - ] - } -} -``` - -**Getting Request Data** - - - -In the `on_cmd` method, the first step is to retrieve the request data, which is the property in the cmd. We define a `request_t` class to represent the request data. - -Create a file called `model.h` in the `include` directory of your extension project with the following content: - -``` -#pragma once - -#include "nlohmann/json.hpp" -#include -#include - -namespace first_cxx_extension_extension { - -class request_payload_t { -public: - friend void from_json(const nlohmann::json &j, request_payload_t &payload); - - friend class request_t; - -private: - uint8_t err_no; - std::string err_msg; -}; - -class request_t { -public: - friend void from_json(const nlohmann::json &j, request_t &request); - -private: - std::string app_id; - int8_t client_type; - request_payload_t payload; -}; - -} // namespace first_cxx_extension_extension -``` - -In the `src` directory, create a file called `model.cc` with the following content: - -``` -#include "model.h" - -namespace first_cxx_extension_extension { -void from_json(const nlohmann::json &j, request_payload_t &payload) { - if (j.contains("err_no")) { - j.at("err_no").get_to(payload.err_no); - } - - if (j.contains("err_msg")) { - j.at("err_msg").get_to(payload.err_msg); - } -} - -void from_json(const nlohmann::json &j, request_t &request) { - if (j.contains("app_id")) { - j.at("app_id").get_to(request.app_id); - } - - if (j.contains("client_type")) { - j.at("client_type").get_to(request.client_type); - } - - if (j.contains("payload")) { - j.at("payload").get_to(request.payload); - } -} -} // namespace first_cxx_extension_extension -``` - -To parse the request data, you can use the `get_property` API provided by TEN. Here is an example of how to implement it: - -``` -// model.h - -class request_t { -public: - void from_cmd(rte::cmd_t &cmd); - - // ... -} - -// model.cc - -void request_t::from_cmd(rte::cmd_t &cmd) { - app_id = cmd.get_property_string("app_id"); - client_type = cmd.get_property_int8("client_type"); - - auto payload_str = cmd.get_property_to_json("payload"); - if (!payload_str.empty()) { - auto payload_json = nlohmann::json::parse(payload_str); - from_json(payload_json, payload); - } -} -``` - -To return a response, you need to create a `cmd_result_t` object and set the properties accordingly. Then, pass the `cmd_result_t` object to TEN runtime to return it to the requester. Here is an example: - -``` -// model.h - -class request_t { -public: - bool validate(std::string *err_msg) { - if (app_id.length() < 64) { - *err_msg = "invalid app_id"; - return false; - } - - return true; - } -} - -// main.cc - -void on_cmd(rte::rte_t &rte, std::unique_ptr cmd) override { - request_t request; - request.from_cmd(*cmd); - - std::string err_msg; - if (!request.validate(&err_msg)) { - auto result = rte::cmd_result_t::create(RTE_STATUS_CODE_ERROR); - result->set_property("err_no", 1); - result->set_property("err_msg", err_msg.c_str()); - - rte.return_result(std::move(result), std::move(cmd)); - } -} -``` - -In the example above, `rte::cmd_result_t::create` is used to create a `cmd_result_t` object with an error code. `result.set_property` is used to set the properties of the `cmd_result_t` object. Finally, `rte.return_result` is called to return the `cmd_result_t` object to the requester. - -**Passing Requests to Downstream Extensions** - - - -If an extension needs to send a message to another extension, it can call the `send_cmd()` API. Here is an example: - -``` -void on_cmd(rte::rte_t &rte, std::unique_ptr cmd) override { - request_t request; - request.from_cmd(*cmd); - - std::string err_msg; - if (!request.validate(&err_msg)) { - // ... - } else { - rte.send_cmd(std::move(cmd)); - } -} -``` - -The first parameter in `send_cmd()` is the command of the request, and the second parameter is the handler for the returned `cmd_result_t`. The second parameter can also be omitted, indicating that no special handling is required for the returned result. If the command was originally sent from a higher-level extension, the runtime will automatically return it to the upper-level extension. - -Developers can also pass a response handler, like this: - -``` -rte.send_cmd( - std::move(cmd), - [](rte::rte_t &rte, std::unique_ptr result) { - rte.return_result_directly(std::move(result)); - }); -``` - -In the example above, the `return_result_directly()` method is used in the response handler. You can see that this method differs from `return_result()` in that it does not pass the original command object. This is mainly because: - -* For TEN message objects (cmd/data/pcm\_frame/image\_frame), ownership is transferred to the extension in the message callback method, such as `on_cmd()`. This means that once the extension receives the command, the TEN runtime will not perform any read/write operations on it. When the extension calls the `send_cmd()` or `return_result()` API, it means that the extension is returning the ownership of the command back to the TEN runtime for further processing, such as message delivery. After that, the extension should not perform any read/write operations on the command. -* The `result` in the response handler (i.e., the second parameter of `send_cmd()`) is returned by the downstream extension, and at this point, the result is already bound to the command, meaning that the runtime has the return path information for the result. Therefore, there is no need to pass the command object again. - -Of course, developers can also process the result in the response handler. - -So far, an example of a simple command processing logic is complete. For other message types such as data, you can refer to the TEN API documentation. - -### Deploying Locally to an App for Integration Testing - - - -arpm provides the ability to publish to a local registry, allowing you to perform integration testing locally without uploading the extension to the central repository. Unlike GO extensions, for C++ extensions, there are no strict requirements on the app's programming language. It can be GO, C++, or Python. - -The deployment process may vary for different apps. The specific steps are as follows: - -* Set up the arpm local registry. -* Upload the extension to the local registry. -* Download the app from the central repository (default\_app\_cpp/default\_app\_go) for integration testing. -* For C++ apps: - * Install the first\_cxx\_extension in the app directory. - * Compile in the app directory. At this point, both the app and the extension will be compiled into the out/linux/x64/app/default\_app\_cpp directory. - * Install the required dependencies in out/linux/x64/app/default\_app\_cpp. The working directory for testing is the current directory. -* For GO apps: - * Install the first\_cxx\_extension in the app directory. - * Compile in the addon/extension/first\_cxx\_extension directory, as the GO and C++ compilation toolchains are different. - * Install the dependencies in the app directory. The working directory for testing is the app directory. -* Configure the graph in the app's manifest.json, specifying the recipient of the message as first\_cxx\_extension, and send test messages. - -#### Uploading the Extension to the Local Registry - - - -First, create a temporary config.json file to set up the arpm local registry. For example, the contents of /tmp/code/config.json are as follows: - -``` -{ - "registry": [ - "file:///tmp/code/repository" - ] -} -``` - -This sets the local directory /tmp/code/repository as the arpm local registry. - -Note - -* Be careful not to place it in \~/.arpm/config.json, as it will affect the subsequent download of dependencies from the central repository. - -Then, in the first\_cxx\_extension directory, execute the following command to upload the extension to the local registry: - -``` -$ arpm --config-file /tmp/code/config.json publish -``` - -After the command completes, the uploaded extension can be found in the /tmp/code/repository/extension/first\_cxx\_extension/0.1.0 directory. - -#### Prepare app for testing (C++) - - - -1. Install default\_app\_cpp as the test app in an empty directory. - -> ``` -> $ arpm install app default_app_cpp -> ``` -> -> After the command is successfully executed, there will be a directory named default\_app\_cpp in the current directory. -> -> Note -> -> * When installing an app, its dependencies will be automatically installed. - -2. Install first\_cxx\_extension that we want to test in the app directory. - -> Execute the following command: -> -> ``` -> $ arpm --config-file /tmp/code/config.json install extension first_cxx_extension -> ``` -> -> After the command is completed, there will be a first\_cxx\_extension directory in the addon/extension directory. -> -> Note -> -> * It is important to note that since first\_cxx\_extension is in the local registry, the configuration file path with the local registry specified by --config-file needs to be the same as when publishing. - -3. Add an extension as a message producer. - -> first\_cxx\_extension is expected to receive a hello cmd, so we need a message producer. One way is to add an extension as a message producer. To conveniently generate test messages, an http server can be integrated into the producer's extension. -> -> First, create an http server extension based on default\_extension\_cpp. Execute the following command in the app directory: -> -> ``` -> $ arpm install extension default_extension_cpp --template-mode --template-data package_name=http_server -> ``` -> -> The main functionality of the http server is: -> -> * Start a thread running the http server in the extension's on\_start(). -> * Convert incoming requests into TEN cmds named hello and send them using send\_cmd(). -> * Expect to receive a cmd\_result\_t response and write its content to the http response. -> -> Here, we use cpp-httplib ([https://github.com/yhirose/cpp-httplib](https://github.com/yhirose/cpp-httplib)) as the implementation of the http server. -> -> First, download httplib.h and place it in the include directory of the extension. Then, add the implementation of the http server in src/main.cc. Here is an example code: -> -> ``` -> #include "httplib.h" -> #include "nlohmann/json.hpp" -> #include "rte_runtime/binding/cpp/rte.h" -> -> namespace http_server_extension { -> -> class http_server_extension_t : public rte::extension_t { -> public: -> explicit http_server_extension_t(const std::string &name) -> : extension_t(name) {} -> -> void on_start(rte::rte_t &rte) override { -> rte_proxy = rte::rte_proxy_t::create(rte); -> srv_thread = std::thread([this] { -> server.Get("/health", -> [](const httplib::Request &req, httplib::Response &res) { -> res.set_content("OK", "text/plain"); -> }); -> -> // Post handler, receive json body. -> server.Post("/hello", [this](const httplib::Request &req, -> httplib::Response &res) { -> // Receive json body. -> auto body = nlohmann::json::parse(req.body); -> body["rte"]["name"] = "hello"; -> -> auto cmd = rte::cmd_t::create_from_json(body.dump().c_str()); -> auto cmd_shared = -> std::make_shared>(std::move(cmd)); -> -> std::condition_variable *cv = new std::condition_variable(); -> -> auto response_body = std::make_shared(); -> -> rte_proxy->notify([cmd_shared, response_body, cv](rte::rte_t &rte) { -> rte.send_cmd( -> std::move(*cmd_shared), -> [response_body, cv](rte::rte_t &rte, -> std::unique_ptr result) { -> auto err_no = result->get_property_uint8("err_no"); -> if (err_no > 0) { -> auto err_msg = result->get_property_string("err_msg"); -> response_body->append(err_msg); -> } else { -> response_body->append("OK"); -> } -> -> cv->notify_one(); -> }); -> }); -> -> std::unique_lock lk(mtx); -> cv->wait(lk); -> delete cv; -> -> res.set_content(response_body->c_str(), "text/plain"); -> }); -> -> server.listen("0.0.0.0", 8001); -> }); -> -> rte.on_start_done(); -> } -> -> void on_stop(rte::rte_t &rte) override { -> // Extension stop. -> -> server.stop(); -> srv_thread.join(); -> delete rte_proxy; -> -> rte.on_stop_done(); -> } -> -> private: -> httplib::Server server; -> std::thread srv_thread; -> rte::rte_proxy_t *rte_proxy{nullptr}; -> std::mutex mtx; -> }; -> -> RTE_CPP_REGISTER_ADDON_AS_EXTENSION(http_server, http_server_extension_t); -> -> } // namespace http_server_extension -> ``` - -Here, a new thread is created in `on_start()` to run the http server because we don't want to block the extension thread. This way, the converted cmd requests are generated and sent from `srv_thread`. In the TEN runtime, to ensure thread safety, we use `rte_proxy_t` to pass calls like `send_cmd()` from threads outside the extension thread. - -This code also demonstrates how to clean up external resources in `on_stop()`. For an extension, you should release the `rte_proxy_t` before `on_stop_done()`, which stops the external thread. - -1. Configure the graph. - -In the app's `manifest.json`, configure `predefined_graph` to specify that the `hello` cmd generated by `http_server` should be sent to `first_cxx_extension`. For example: - -> ``` -> "predefined_graphs": [ -> { -> "name": "testing", -> "auto_start": true, -> "nodes": [ -> { -> "type": "extension_group", -> "name": "http_thread", -> "addon": "default_extension_group" -> }, -> { -> "type": "extension", -> "name": "http_server", -> "addon": "http_server", -> "extension_group": "http_thread" -> }, -> { -> "type": "extension", -> "name": "first_cxx_extension", -> "addon": "first_cxx_extension", -> "extension_group": "http_thread" -> } -> ], -> "connections": [ -> { -> "extension_group": "http_thread", -> "extension": "http_server", -> "cmd": [ -> { -> "name": "hello", -> "dest": [ -> { -> "extension_group": "http_thread", -> "extension": "first_cxx_extension" -> } -> ] -> } -> ] -> } -> ] -> } -> ] -> ``` - -5. Compile the app. - -> Execute the following commands in the app directory: -> -> ``` -> $ ag gen linux x64 debug -> $ ag build linux x64 debug -> ``` -> -> After the compilation is complete, the compilation output for the app and extension will be generated in the directory out/linux/x64/app/default\_app\_cpp. -> -> However, it cannot be run directly at this point as it is missing the dependencies of the extension group. - -6. Install the extension group. - -> Switch to the compilation output directory. -> -> ``` -> $ cd out/linux/x64/app/default_app_cpp -> ``` -> -> Install the extension group. -> -> ``` -> $ arpm install extension_group default_extension_group -> ``` - -7. Start the app. - -> In the compilation output directory, execute the following command: -> -> ``` -> $ ./bin/default_app_cpp -> ``` -> -> After the app starts, you can now test it by sending messages to the http server. For example, use curl to send a request with an invalid app\_id: -> -> ``` -> $ curl --location 'http://127.0.0.1:8001/hello' \ -> --header 'Content-Type: application/json' \ -> --data '{ -> "app_id": "123", -> "client_type": 1, -> "payload": { -> "err_no": 0 -> } -> }' -> ``` -> -> The expected response should be "invalid app\_id". - -### Debugging extension in an app - -#### App (C++) - -A C++ app is compiled into an executable file with the correct `rpath` set. Therefore, debugging a C++ app only requires adding the following configuration to `.vscode/launch.json`: - -``` -"configurations": [ - { - "name": "App (C/C++) (lldb, launch)", - "type": "lldb", - "request": "launch", - "program": "${workspaceFolder}/out/linux/x64/app/default_app_cpp/bin/default_app_cpp", - "args": [], - "cwd": "${workspaceFolder}/out/linux/x64/app/default_app_cpp" - } - ] -``` diff --git a/tutorials/how-to-build-extension-with-go-beta.md b/tutorials/how-to-build-extension-with-go-beta.md deleted file mode 100644 index 40bd11d..0000000 --- a/tutorials/how-to-build-extension-with-go-beta.md +++ /dev/null @@ -1,1120 +0,0 @@ ---- -hidden: true -layout: - title: - visible: true - description: - visible: false - tableOfContents: - visible: true - outline: - visible: true - pagination: - visible: true ---- - -# 🚧 How to build extension with Go(beta) - -## Overview - -Introduction to developing a TEN extension using the GO language, as well as debugging and deploying it to run in an app. - -This tutorial includes the following: - -* How to create a GO extension development project using `arpm`. -* How to use TEN API to implement the functionality of the extension, such as sending and receiving messages. -* How to configure `cgo` if needed. -* How to write unit tests and debug code. -* How to deploy the extension locally in an app for integration testing. -* How to debug extension code in an app. - -{% hint style="info" %} -Unless otherwise specified, the commands and code in this tutorial are executed in a Linux environment. Similar steps can be followed for other platforms. -{% endhint %} - - - -## Prerequisites - -Download the latest version of arpm and configure the PATH. You can check if it is configured correctly by running the following command: - -```bash -$ arpm -h -``` - -If the configuration is correct, it will display the help information for arpm. - -* Install GO 1.20 or above, preferably the latest version. - -Note: - -* TEN GO API uses cgo, so make sure cgo is enabled by default. You can check by running the following command: - -``` -$ go env CGO_ENABLED -``` - -If it returns 1, it means cgo is enabled by default. Otherwise, you can enable cgo by running the following command: - -``` -$ go env -w CGO_ENABLED=1 -``` - -### Creating a GO Extension Project - - - -A GO extension is essentially a go module project that includes the necessary dependencies and configuration files to meet the requirements of a TEN extension. TEN provides a default GO extension template project that developers can use to quickly create a GO extension project. - -#### Creating from Template - - - -To create a project named "first\_go\_extension" based on the default\_extension\_go template, use the following command: - -``` -$ arpm install extension default_extension_go --template-mode --template-data package_name=first_go_extension -``` - -After executing the command, a directory named "first\_go\_extension" will be created in the current directory. This directory will contain the GO extension project with the following structure: - -``` -. -β”œβ”€β”€ default_extension.go -β”œβ”€β”€ go.mod -β”œβ”€β”€ manifest.json -└── property.json -``` - -In this structure: - -* "default\_extension.go" contains a simple extension implementation that includes calls to the TEN GO API. The usage of the TEN API will be explained in the next section. -* "manifest.json" and "property.json" are the standard configuration files for TEN extensions. "manifest.json" is used to declare metadata information such as the version, dependencies, and schema definition of the extension. "property.json" is used to declare the business configuration of the extension. - -The `property.json` file is initially empty and The `manifest.json` file includes a dependency on "rte\_runtime\_go" by default: - -``` -{ - "type": "extension", - "name": "first_go_extension", - "version": "0.1.0", - "language": "go", - "dependencies": [ - { - "type": "system", - "name": "rte_runtime_go", - "version": "0.2.0" - } - ], - "api": {} -} -``` - -Note - -* Please note that according to TEN's naming convention, the name should be alphanumeric because when integrating the extension into an app, a directory will be created based on the extension name. Additionally, TEN will provide functionality to load the manifest.json and property.json files from the extension directory. -* The dependencies section declares the dependencies of the current extension. When installing the TEN package, arpm will automatically download the declared dependencies. -* The api section is used to declare the schema of the extension. For the usage of schema, refer to: [rte-schema](https://github.com/rte-design/ASTRA.ai/blob/main/docs/rte-schema.md) - -TEN GO API is not publicly available and needs to be installed locally using arpm. Therefore, the go.mod file uses the replace directive to reference the TEN GO API. For example: - -``` -replace agora.io/rte => ../../../interface -``` - -When the extension is installed in an app, it will be placed in the addon/extension/ directory. At the same time, the TEN GO API will be installed in the root directory of the app. The expected directory structure is as follows: - -``` -. -β”œβ”€β”€ addon -β”‚ └── extension -β”‚ └── first_go_extension -β”‚ β”œβ”€β”€ default_extension.go -β”‚ β”œβ”€β”€ go.mod -β”‚ β”œβ”€β”€ manifest.json -β”‚ └── property.json -β”œβ”€β”€ go.mod -β”œβ”€β”€ go.sum -β”œβ”€β”€ interface -β”‚ └── rtego -└── main.go -``` - -Therefore, the replace directive in the extension's go.mod file points to the interface directory in the app. - -#### Manual Creation - - - -Alternatively, developers can create a go module project using the go init command and then create the manifest.json and property.json files based on the examples provided above. - -Do not add the dependency on the TEN GO API yet because the required interface directory is not available locally. It needs to be installed using arpm before adding the dependency. - -To convert a newly created go module project or an existing one into an extension project, follow these steps: - -* Create the property.json file in the project directory and add the necessary configuration for the extension. -* Create the manifest.json file in the project directory and specify the type, name, version, language, and dependencies information. Note that these fields are required. - * The type should be extension. - * The language should be go. - * The dependencies should include the dependency on rte\_runtime\_go and any other dependencies as needed. - -### Download Dependencies - - - -Execute the following command in the extension project directory to download dependencies: - -``` -$ arpm install -``` - -After the command is successfully executed, a .rte directory will be generated in the current directory, which contains all the dependencies of the current extension. - -Note - -* There are two modes for an extension: development mode and runtime mode. In development mode, the root directory is the source code directory of the extension. In runtime mode, the root directory is the app directory. Therefore, the placement path of dependencies is different in these two modes. The .rte directory mentioned here is the root directory of dependencies in development mode. - -The directory structure is as follows: - -``` -β”œβ”€β”€ default_extension.go -β”œβ”€β”€ go.mod -β”œβ”€β”€ manifest.json -β”œβ”€β”€ property.json -└── .rte - └── app - -``` - -In this structure, .rte/app/interface is the module for the TEN GO API. - -Therefore, in development mode, the go.mod file of the extension should be modified as follows: - -``` -replace agora.io/rte => ./.rte/app/interface -``` - -If you manually created the extension as mentioned in the previous section, you also need to execute the following command in the extension directory: - -``` -$ go get agora.io/rte -``` - -The expected output should be: - -``` -go: added agora.io/rte v0.0.0-00010101000000-000000000000 -``` - -At this point, a TEN GO extension project has been created. - -### Implementing Extension Functionality - - - -The "go.mod" file uses the "replace" directive to reference the TEN GO API: - -``` -replace agora.io/rte => ../../../interface -``` - -When the extension is installed in an app, it will be placed in the "addon/extension/" directory. The TEN GO API will be installed in the root directory of the app. The expected directory structure is as follows: - -``` -. -β”œβ”€β”€ addon -β”‚ └── extension -β”‚ └── first_go_extension -β”‚ β”œβ”€β”€ default_extension.go -β”‚ β”œβ”€β”€ go.mod -β”‚ β”œβ”€β”€ manifest.json -β”‚ └── property.json -β”œβ”€β”€ go.mod -β”œβ”€β”€ go.sum -β”œβ”€β”€ interface -β”‚ └── rtego -└── main.go -``` - -Therefore, the "replace" directive in the extension's go.mod file points to the "interface" directory in the app. - -#### Manual Creation - - - -Alternatively, developers can manually create a go module project and then add the "manifest.json" and "property.json" files based on the examples provided above. - -For a newly created go module project or an existing one, to convert it into an extension project, follow these steps: - -* Create the "property.json" file in the project directory and add the necessary configuration for the extension. -* Create the "manifest.json" file in the project directory and specify the "type", "name", "version", "language", and "dependencies" information. Note that these fields are required. - * The "type" should be "extension". - * The "language" should be "go". - * The "dependencies" should include the dependency on "rte\_runtime\_go" and any other dependencies as needed. - -### Download Dependencies - - - -Execute the following command in the extension project directory to download dependencies: - -``` -$ arpm install -``` - -After the command is successfully executed, a `.rte` directory will be generated in the current directory, which contains all the dependencies of the current extension. - -Note: - -* There are two modes for an extension: development mode and runtime mode. In development mode, the root directory is the source code directory of the extension. In runtime mode, the root directory is the app directory. Therefore, the placement path of dependencies is different in these two modes. The `.rte` directory mentioned here is the root directory of dependencies in development mode. - -The directory structure is as follows: - -``` -β”œβ”€β”€ default_extension.go -β”œβ”€β”€ go.mod -β”œβ”€β”€ manifest.json -β”œβ”€β”€ property.json -└── .rte - └── app - β”œβ”€β”€ addon - β”œβ”€β”€ include - β”œβ”€β”€ interface - └── lib -``` - -In this structure, `.rte/app/interface` is the module for the TEN GO API. - -Therefore, in development mode, the `go.mod` file of the extension should be modified as follows: - -``` -replace agora.io/rte => ./.rte/app/interface -``` - -If you manually created the extension as mentioned in the previous section, you also need to execute the following command in the extension directory: - -``` -$ go get agora.io/rte -``` - -The expected output should be: - -``` -go: added agora.io/rte v0.0.0-00010101000000-000000000000 -``` - -At this point, a TEN GO extension project has been created. - -### Implementing Extension Functionality - - - -For developers, there are two things to do: - -* Create an extension as a channel for interaction with TEN runtime. -* Register the extension as an addon (referred to as addon in TEN) to use the extension in the graph declaratively. - -#### Create extension struct - - - -The extension created by developers needs to implement the `rtego.Extension` interface, which is defined as follows: - -``` -type Extension interface { - OnInit( - rte Rte, - manifest MetadataInfo, - property MetadataInfo, - ) - OnStart(rte Rte) - OnStop(rte Rte) - OnDeinit(rte Rte) - OnCmd(rte Rte, cmd Cmd) - OnData(rte Rte, data Data) - OnImageFrame(rte Rte, imageFrame ImageFrame) - OnPcmFrame(rte Rte, pcmFrame PcmFrame) -} -``` - -It includes four lifecycle functions and four message handling functions: - -Lifecycle functions: - -* OnInit: Used to initialize the extension instance, such as setting the extension's configuration. -* OnStart: Used to start the extension instance, such as creating connections to external services. The extension will not receive messages until it is started. In OnStart, you can use the `rte.GetProperty` related APIs to get the extension's configuration. -* OnStop: Used to stop the extension instance, such as closing connections to external services. -* OnDeinit: Used to destroy the extension instance, such as releasing memory resources. - -Message handling functions: - -* OnCmd/OnData/OnImageFrame/OnPcmFrame: Callback functions for receiving four types of messages. The message types in TEN can be referred to as [message-type-and-name](https://github.com/rte-design/ASTRA.ai/blob/main/docs/message-type-and-name.md) - -For the implementation of the extension, you may only need to focus on a subset of message types. To facilitate implementation, TEN provides a default `DefaultExtension`. Developers have two options: either directly implement the `rtego.Extension` interface or embed `DefaultExtension` and override the necessary methods. - -For example, the `default_extension_go` template uses the approach of embedding `DefaultExtension`. Here is an example: - -``` -type defaultExtension struct { - rtego.DefaultExtension -} -``` - -#### Register extension - - - -After defining the extension, it needs to be registered as an addon in the TEN runtime. For example, the registration code in `default_extension.go` is as follows: - -``` -func init() { - // Register addon - rtego.RegisterAddonAsExtension( - "default_extension_go", - rtego.NewDefaultExtensionAddon(newDefaultExtension), - ) -} -``` - -* `RegisterAddonAsExtension` is the method to register an addon object as an TEN extension addon. The addon types in TEN also include extension\_group and protocol, which will be covered in the subsequent integration testing section. - * The first parameter is the addon name, which is a unique identifier for the addon. It will be used to define the extension in the graph declaratively. For example: - - ``` - { - "nodes": [ - { - "type": "extension", - "name": "extension_go", - "addon": "default_extension_go", - "extension_group": "default" - } - ] - } - ``` - - In this example, it means using an addon named `default_extension_go` to create an extension instance named `extension_go`. - * The second parameter is an addon object. TEN provides a simple way to create it - `NewDefaultExtensionAddon`, which takes the constructor of the business extension as a parameter. For example: - - ``` - func newDefaultExtension(name string) rtego.Extension { - return &defaultExtension{} - } - ``` - -Note - -* It is important to note that the addon name must be unique because in the graph, the addon name is used as a unique index to find the implementation. Here, change the first parameter to `first_go_extension`. - -#### OnInit - - - -Developers can set the extension's configuration in OnInit(), as shown below: - -``` -func (p *defaultExtension) OnInit(rte rtego.Rte, property rtego.MetadataInfo, manifest rtego.MetadataInfo) { - property.Set(rtego.MetadataTypeJSONFileName, "customized_property.json") - rte.OnInitDone() -} -``` - -* Both `property` and `manifest` can customize the configuration content using the `Set()` method. In the example, the first parameter `rtego.MetadataTypeJSONFileName` indicates that the custom property exists as a local file, and the second parameter is the file path relative to the extension directory. So in the example, when the app loads the extension, it will load `\/addon/extension/first_go_extension/customized_property.json`. -* TEN OnInit provides default configuration loading logic - if developers do not call `property.Set()`, the `property.json` in the extension directory will be loaded by default. Similarly, if `manifest.Set()` is not called, the `manifest.json` in the extension directory will be loaded by default. Also, if developers call `property.Set()`, the `property.json` will not be loaded by default. -* OnInit is an asynchronous method, and developers need to call `rte.OnInitDone()` to inform the TEN runtime that the initialization is expected to be completed. - -Note - -* Please note that `OnInitDone()` is also an asynchronous method, which means that even after `OnInitDone()` returns, developers still cannot use `rte.GetProperty()` to get the configuration. For the extension, you need to wait until the `OnStart()` callback method. - -#### OnStart - - - -When OnStart is called, it indicates that OnInitDone() has already been executed and the extension's property has been loaded. From this point on, the extension can access the configuration. Here is an example: - -``` -func (p *defaultExtension) OnStart(rte rtego.Rte) { - prop, err := rte.GetPropertyString("some_string") - - if err != nil { - // handle error. - } else { - // do something. - } - - rte.OnStartDone() -} -``` - -`rte.GetPropertyString()` is used to retrieve a property of type string with the name "some\_string". If the property does not exist or the type does not match, an error will be returned. If the extension's configuration is as follows: - -``` -{ - "some_string": "hello world" -} -``` - -Then the value of `prop` will be "hello world". - -Similar to OnInit, OnStart is also an asynchronous method, and developers need to call `rte.OnStartDone()` to inform the TEN runtime that the start process is expected to be completed. - -For error handling, as shown in the previous example, `rte.GetPropertyString()` returns an error. For TEN API, errors are generally returned as `rtego.RteError` type. Therefore, you can handle the error as follows: - -``` -func (p *defaultExtension) OnStart(rte rtego.Rte) { - prop, err := rte.GetPropertyString("some_string") - - if err != nil { - // handle error. - var rteErr *rtego.RteError - if errors.As(err, &rteErr) { - log.Printf("Failed to get property, cause: %s.\n", rteErr.ErrMsg()) - } - } else { - // do something. - } - - rte.OnStartDone() -} -``` - -`rtego.RteError` provides the `ErrMsg()` method to retrieve the error message and the `ErrNo()` method to retrieve the error code. - -TEN provides four types of messages: Cmd, Data, ImageFrame, and PcmFrame. Developers can handle these four types of messages by implementing the OnCmd, OnData, OnImageFrame, and OnPcmFrame callback methods. - -Taking Cmd as an example, let's see how to receive and send messages. - -Assume that the first\_go\_extension will receive a Cmd named "hello" with the following properties: - -| name | type | -| ---------------- | ------ | -| app\_id | string | -| client\_type | int8 | -| payload | object | -| payload.err\_no | uint8 | -| payload.err\_msg | string | - -The processing logic of the first\_go\_extension for the "hello" Cmd is as follows: - -* If the app\_id or client\_type parameters are invalid, return an error: - -``` -{ - "err_no": 1001, - "err_msg": "Invalid argument." -} -``` - -* If payload.err\_no is greater than 0, return an error with the content from the payload. -* If payload.err\_no is equal to 0, forward the "hello" Cmd downstream for further processing. After receiving the processing result from the downstream extension, return the result. - -To describe the behavior of the extension in the manifest.json file, you can specify: - -* What messages the extension will receive, including the message name and the structure definition of the properties (schema). -* What messages the extension will send, including the message name and the structure definition of the properties. -* For Cmd messages, a response definition is required (referred to as Result in TEN). - -With these definitions, the TEN runtime will perform validation based on the schema before delivering messages to the extension and before the extension sends messages through the TEN runtime. This ensures the validity of the messages and facilitates the understanding of the extension's protocol by its users. - -The schema is defined in the api field of the manifest.json file. cmd\_in defines the Cmd messages that the extension will receive, and cmd\_out defines the Cmd messages that the extension will send. - -Here is an example of the manifest.json for the first\_go\_extension: - -``` -{ - "type": "extension", - "name": "first_go_extension", - "version": "0.1.0", - "language": "go", - "dependencies": [ - { - "type": "system", - "name": "rte_runtime_go", - "version": "0.2.0" - } - ], - "api": { - "cmd_in": [ - { - "name": "hello", - "property": { - "app_id": { - "type": "string" - }, - "client_type": { - "type": "int8" - }, - "payload": { - "type": "object", - "properties": { - "err_no": { - "type": "uint8" - }, - "err_msg": { - "type": "string" - } - } - } - }, - "required": ["app_id", "client_type"], - "result": { - "property": { - "err_no": { - "type": "uint8" - }, - "err_msg": { - "type": "string" - } - }, - "required": ["err_no"] - } - } - ], - "cmd_out": [ - { - "name": "hello", - "property": { - "app_id": { - "type": "string" - }, - "client_type": { - "type": "string" - }, - "payload": { - "type": "object", - "properties": { - "err_no": { - "type": "uint8" - }, - "err_msg": { - "type": "string" - } - } - } - }, - "required": ["app_id", "client_type"], - "result": { - "property": { - "err_no": { - "type": "uint8" - }, - "err_msg": { - "type": "string" - } - }, - "required": ["err_no"] - } - } - ] - } -} -``` - -**Getting Request Data** - - - -In the `OnCmd` method, the first step is to retrieve the request data, which is the property of the Cmd object. We define a `Request` struct to represent the request data as follows: - -``` -type RequestPayload struct { - ErrNo uint8 `json:"err_no"` - ErrMsg string `json:"err_msg"` -} - -type Request struct { - AppID string `json:"app_id"` - ClientType int8 `json:"client_type"` - Payload RequestPayload `json:"payload"` -} -``` - -For TEN message objects like Cmd, Data, PcmFrame, and ImageFrame, properties can be set. TEN provides getter/setter APIs for properties. The logic to retrieve the request data is to use the `GetProperty` API to parse the property of the Cmd object, as shown below: - -``` -func parseRequestFromCmdProperty(cmd rtego.Cmd) (*Request, error) { - request := &Request{} - - if appID, err := cmd.GetPropertyString("app_id"); err != nil { - return nil, err - } else { - request.AppID = appID - } - - if clientType, err := cmd.GetPropertyInt8("client_type"); err != nil { - return nil, err - } else { - request.ClientType = clientType - } - - if payloadBytes, err := cmd.GetPropertyToJSONBytes("payload"); err != nil { - return nil, err - } else { - err := json.Unmarshal(payloadBytes, &request.Payload) - - rtego.ReleaseBytes(payloadBytes) - - if err != nil { - return nil, err - } - } - - return request, nil -} -``` - -* `GetPropertyString()` and `GetPropertyInt8()` are specialized APIs to retrieve properties of specific types. For example, `GetPropertyString()` expects the property to be of type string, and if it's not, an error will be returned. -* `GetPropertyToJSONBytes()` expects the value of the property to be a JSON-serialized data. This API is provided because TEN runtime does not expect to be bound to a specific JSON library. After obtaining the property as a slice, developers can choose a JSON library for deserialization as needed. -* `rtego.ReleaseBytes()` is used to release the `payloadBytes` because TEN GO binding layer provides a memory pool. - -**Returning a Response** - - - -After parsing the request data, we can now implement the first step of the processing flow - returning an error response if the parameters are invalid. In TEN extensions, a response is represented by a `CmdResult`. So, returning a response involves the following two steps: - -* Creating a `CmdResult` object and setting properties as needed. -* Handing over the created `CmdResult` object to the TEN runtime, which will handle returning it based on the path of the requester. - -Here's the implementation: - -``` -const InvalidArgument uint8 = 1 - -func (r *Request) validate() error { - if len(r.AppID) < 64 { - return errors.New("invalid app_id") - } - - if r.ClientType != 1 { - return errors.New("invalid client_type") - } - - return nil -} - -func (p *defaultExtension) OnCmd( - rte rtego.Rte, - cmd rtego.Cmd, -) { - request, err := parseRequestFromCmdProperty(cmd) - if err == nil { - err = request.validate() - } - - if err != nil { - result, fatal := rtego.NewCmdResult(rtego.Error) - if fatal != nil { - log.Fatalf("Failed to create result, %v\n", fatal) - return - } - - result.SetProperty("err_no", InvalidArgument) - result.SetPropertyString("err_msg", err.Error()) - - rte.ReturnResult(result, cmd) - } -} -``` - -* `rtego.NewCmdResult()` is used to create a `CmdResult` object. The first parameter is the error code - either Ok or Error. This error code is built-in to the TEN runtime and indicates whether the processing was successful. Developers can also use `GetStatusCode()` to retrieve this error code. Additionally, developers can define more detailed business-specific error codes, as shown in the example. -* `result.SetProperty()` is used to set properties in the `CmdResult` object. Properties are set as key-value pairs. `SetPropertyString()` is a specialized API of `SetProperty()` and is provided to reduce the performance overhead of passing GO strings. -* `rte.ReturnResult()` is used to return the `CmdResult` to the requester. The first parameter is the response, and the second parameter is the request. The TEN runtime will handle returning the response to the requester based on the request's path. - -**Passing the Request to Downstream Extensions** - - - -If an extension wants to send a message to another extension, it can call the `SendCmd()` API. Here's an example: - -``` -func (p *defaultExtension) OnCmd( - rte rtego.Rte, - cmd rtego.Cmd, -) { - // ... - - if err != nil { - // ... - } else { - // Dispatch the request to the upstream. - rte.SendCmd(cmd, func(r rtego.Rte, result rtego.CmdResult) { - r.ReturnResultDirectly(result) - }) - } -} -``` - -* The first parameter in `SendCmd()` is the command of the request, and the second parameter is the callback function that handles the `CmdResult` returned by the downstream. The second parameter can also be set to `nil`, indicating that no special handling is required for the result returned by the downstream. If the original command was sent from a higher-level extension, the runtime will automatically return it to the upper-level extension. -* In the example's callback function, `ReturnResultDirectly()` is used. You can see that this method differs from `ReturnResult()` in that it does not pass the original command object. This is mainly because: - * For TEN message objects like Cmd/Data/PcmFrame/ImageFrame, there is an ownership concept. In the extension's message callback method, such as `OnCmd()`, the TEN runtime transfers ownership of the Cmd object to the extension. This means that once the extension receives the Cmd, the TEN runtime will not perform any read or write operations on it. When the extension calls the `SendCmd()` or `ReturnResult()` API, it returns ownership of the Cmd back to the TEN runtime, which handles further processing, such as message delivery. After that, the extension should not perform any read or write operations on the Cmd. - * The `result` in the response handler (the second parameter of `SendCmd()`) is returned by the downstream, and at this point, the result is already bound to the Cmd, meaning the runtime has information about the result's return path. Therefore, there is no need to pass the Cmd object again. - -Of course, developers can also handle the result in the response handler. - -So far, an example of a simple Cmd processing logic is complete. For other message types like Data, you can refer to the TEN API documentation. - -### Deploying Locally to the App for Integration Testing - - - -arpm provides the ability to publish and use a local registry, allowing you to perform integration testing locally without uploading the extension to the central repository. Here are the steps: - -* Set up the arpm local registry. -* Upload the extension to the local registry. -* Download the default\_app\_go from the central repository as the integration testing environment, along with any other dependencies needed. -* Download the first\_go\_extension from the local registry. -* Configure the graph in default\_app\_go to specify the receiver of the message as first\_go\_extension and send test messages. - -#### Uploading the Extension to the Local Registry - - - -Before uploading, restore the dependency path for TEN GO binding in first\_go\_extension/go.mod because it needs to be installed under the app. For example: - -``` -replace agora.io/rte => ../../../interface -``` - -First, create a temporary config.json file to set up the arpm local registry. For example, the contents of /tmp/code/config.json are as follows: - -``` -{ - "registry": [ - "file:///tmp/code/repository" - ] -} -``` - -This sets the local directory /tmp/code/repository as the arpm local registry. - -Note - -* Make sure not to place it in \~/.arpm/config.json as it will affect downloading dependencies from the central repository. - -Then, in the first\_go\_extension directory, execute the following command to upload the extension to the local registry: - -``` -$ arpm --config-file /tmp/code/config.json publish -``` - -After the command completes, the uploaded extension can be found in the directory /tmp/code/repository/extension/first\_go\_extension/0.1.0. - -#### Prepare the test app - - - -1. Install `default_app_go` as the test app in an empty directory. - -``` -$ arpm install app default_app_go -``` - -After the command is successful, you will have a directory named `default_app_go` in the current directory. - -> **Note** -> -> * Since the extension being tested is written in Go, the app must also be written in Go. `default_app_go` is a Go app template provided by TEN. -> * When installing the app, its dependencies will be automatically installed. - -2. Install the `extension_group` in the app directory. - -Switch to the app directory: - -``` -$ cd default_app_go -``` - -Install the `default_extension_group`: - -``` -$ arpm install extension_group default_extension_group -``` - -After the command completes, you will have a directory named `addon/extension_group/default_extension_group`. - -> **Note** -> -> * `extension_group` is a capability provided by TEN to declare physical threads and specify which extension instances run in which extension groups. `default_extension_group` is the default extension group provided by TEN. - -3. Install the `first_go_extension` that we want to test in the app directory. - -Execute the following command: - -``` -$ arpm --config-file /tmp/code/config.json install extension first_go_extension -``` - -After the command completes, you will have a directory named `addon/extension/first_go_extension`. - -> **Note** -> -> * It is important to note that since `first_go_extension` is in the local registry, you need to specify the configuration file path that contains the local registry configuration using `--config-file`, just like when publishing. - -4. Add an extension as a message producer. - -> The first\_go\_extension is expected to receive a hello cmd, so we need a message producer. One way to achieve this is by adding an extension as a message producer. To conveniently generate test messages, we can integrate an HTTP server into the producer's extension. -> -> First, we can create an HTTP server extension based on default\_extension\_go. Execute the following command in the app directory: -> -> ``` -> $ arpm install extension default_extension_go --template-mode --template-data package_name=http_server -> ``` -> -> Modify the module name in addon/extension/http\_server/go.mod to http\_server. -> -> The main functionalities of the HTTP server are as follows: -> -> * Start an HTTP server in the extension's OnStart() method, running as a goroutine. -> * Convert incoming requests into TEN Cmd with the name hello, and then call SendCmd() to send the message. -> * Expect to receive a CmdResult response and write its content to the HTTP response. -> -> The code implementation is as follows: -> -> ``` -> type defaultExtension struct { -> rtego.DefaultExtension -> rte rtego.Rte -> -> server *http.Server -> } -> -> type RequestPayload struct { -> ErrNo uint8 `json:"err_no"` -> ErrMsg string `json:"err_msg"` -> } -> -> type Request struct { -> AppID string `json:"app_id"` -> ClientType int8 `json:"client_type"` -> Payload RequestPayload `json:"payload"` -> } -> -> func newDefaultExtension(name string) rtego.Extension { -> return &defaultExtension{} -> } -> -> func (p *defaultExtension) defaultHandler(writer http.ResponseWriter, request *http.Request) { -> switch request.URL.Path { -> case "/health": -> writer.WriteHeader(http.StatusOK) -> default: -> resultChan := make(chan rtego.CmdResult, 1) -> -> var req Request -> if err := json.NewDecoder(request.Body).Decode(&req); err != nil { -> writer.WriteHeader(http.StatusBadRequest) -> writer.Write([]byte("Invalid request body.")) -> return -> } -> -> cmd, _ := rtego.NewCmd("hello") -> cmd.SetPropertyString("app_id", req.AppID) -> cmd.SetProperty("client_type", req.ClientType) -> -> payloadBytes, _ := json.Marshal(req.Payload) -> cmd.SetPropertyFromJSONBytes("payload", payloadBytes) -> -> p.rte.SendCmd(cmd, func(rte rtego.Rte, result rtego.CmdResult) { -> resultChan <- result -> }) -> -> result := <-resultChan -> -> writer.WriteHeader(http.StatusOK) -> errNo, _ := result.GetPropertyUint8("err_no") -> -> if errNo > 0 { -> errMsg, _ := result.GetPropertyString("err_msg") -> writer.Write([]byte(errMsg)) -> } else { -> writer.Write([]byte("OK")) -> } -> } -> } -> -> func (p *defaultExtension) OnStart(rte rtego.Rte) { -> p.rte = rte -> -> mux := http.NewServeMux() -> mux.HandleFunc("/", p.defaultHandler) -> -> p.server = &http.Server{ -> Addr: ":8001", -> Handler: mux, -> } -> -> go func() { -> if err := p.server.ListenAndServe(); err != nil { -> if err != http.ErrServerClosed { -> panic(err) -> } -> } -> }() -> -> go func() { -> // Check if the server is ready. -> for { -> resp, err := http.Get("http://127.0.0.1:8001/health") -> if err != nil { -> continue -> } -> -> defer resp.Body.Close() -> -> if resp.StatusCode == 200 { -> break -> } -> -> time.Sleep(50 * time.Millisecond) -> } -> -> fmt.Println("http server starts.") -> -> p.rte.OnStartDone() -> }() -> } -> -> func (p *defaultExtension) OnStop(rte rtego.Rte) { -> fmt.Println("defaultExtension OnStop") -> -> if p.server != nil { -> p.server.Shutdown(context.Background()) -> } -> -> rte.OnStopDone() -> } -> -> func init() { -> fmt.Println("defaultExtension init") -> -> // Register addon -> rtego.RegisterAddonAsExtension( -> "http_server", -> rtego.NewDefaultExtensionAddon(newDefaultExtension), -> ) -> } -> ``` - -1. Configure the graph. - -> In the app's `manifest.json`, configure `predefined_graph` to specify the `hello` cmd generated by `http_server` and send it to `first_go_extension`. For example: - -``` -"predefined_graphs": [ - { - "name": "testing", - "auto_start": true, - "nodes": [ - { - "type": "extension_group", - "name": "http_thread", - "addon": "default_extension_group" - }, - { - "type": "extension", - "name": "http_server", - "addon": "http_server", - "extension_group": "http_thread" - }, - { - "type": "extension", - "name": "first_go_extension", - "addon": "first_go_extension", - "extension_group": "http_thread" - } - ], - "connections": [ - { - "extension_group": "http_thread", - "extension": "http_server", - "cmd": [ - { - "name": "hello", - "dest": [ - { - "extension_group": "http_thread", - "extension": "first_go_extension" - } - ] - } - ] - } - ] - } -] -``` - -6. Compile the app and start it. - -> Run the following command in the app directory: - -``` -$ go run scripts/build/main.go -``` - -After the compilation is complete, an executable file `./bin/main` will be generated by default. - -Start the app by running the following command: - -``` -$ ./bin/main -``` - -When the console outputs the log message "http server starts", it means that the HTTP listening port has been successfully started. You can now send requests to test it. - -For example, use curl to send a request with an invalid `app_id`: - -``` -$ curl --location 'http://127.0.0.1:8001/hello' \ - --header 'Content-Type: application/json' \ - --data '{ - "app_id": "123", - "client_type": 1, - "payload": { - "err_no": 0 - } - }' -``` - -You should expect to receive a response with the message "invalid app\_id". - -### Debugging an extension in the app - - - -The compilation of an TEN app is performed through a script defined by TEN, which includes the necessary configuration settings for compilation. In other words, you cannot compile an TEN app directly using "go build". Therefore, you cannot debug the app using the default method and instead need to choose the "attach" method. - -For example, if you want to debug the app in Visual Studio Code, you can add the following configuration to ".vscode/launch.json": - -``` -{ - "version": "0.2.0", - "configurations": [ - { - "name": "app (golang) (go, attach)", - "type": "go", - "request": "attach", - "mode": "local", - "processId": 0, - "stopOnEntry": true - } - ] -} -``` - -First, compile and start the app using the above method. - -Then, in the "RUN AND DEBUG" window of Visual Studio Code, select "app (golang) (go, attach)" and click "Start Debugging". - -Next, in the pop-up process selection window, locate the running app process and start debugging. - -Note: - -* If you encounter the "the scope of ptrace system call application is limited" error on a Linux environment, you can resolve it by running the following command: - -``` -$ sudo sysctl -w kernel.yama.ptrace_scope=0 -``` - -\ diff --git a/tutorials/how-to-debug-with-logs.md b/tutorials/how-to-debug-with-logs.md deleted file mode 100644 index 8d45841..0000000 --- a/tutorials/how-to-debug-with-logs.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -layout: - title: - visible: true - description: - visible: false - tableOfContents: - visible: true - outline: - visible: true - pagination: - visible: true ---- - -# How to debug with logs - -In this chapter, we’ll cover how to view the logs and understand what they mean. - -For instance, if the Astra AI agent is running at localhost:3000, you might see output similar to the following in the logs: - -
...
-2024/08/13 05:18:38 INFO handlerPing start channelName=agora_aa1aou requestId=435851e4-b5ff-437a-9930-14ae94b1dee7 service=HTTP_SERVER
-2024/08/13 05:18:38 INFO handlerPing end worker="&{ChannelName:agora_aa1aou LogFile:/tmp/astra/app-5ea31f2d5d7a5e48a9dbc583959c1b17-1723525636779876885.log PropertyJsonFile:/tmp/property-5ea31f2d5d7a5e48a9dbc583959c1b17-1723525636779876885.json Pid:245 QuitTimeoutSeconds:60 CreateTs:1723525636 UpdateTs:1723526318}" requestId=435851e4-b5ff-437a-9930-14ae94b1dee7 service=HTTP_SERVER
-[GIN] 2024/08/13 - 05:18:38 | 200 |     2.65725ms |    192.168.65.1 | POST     "/ping"
-...
-
- -The line starting with LogFile: is what we’re interested in. To view the log file, use the following command: - -
cat /tmp/astra/app-5ea31f2d5d7a5e48a9dbc583959c1b17-1723525636779876885.log
-
- -You should then see entries like these: - -{% code title=">_Bash" overflow="wrap" %} -```bash -[SttMs] OnSpeechRecognized -2024/08/13 04:40:43.365841 INFO GetChatCompletionsStream recv for input text: [What's going on? ] sent sentence [Not much,] extension=OPENAI_CHATGPT_EXTENSION -2024/08/13 04:40:43.366019 INFO GetChatCompletionsStream recv for input text: [What's going on? ] first sentence sent, first_sentency_latency 876ms extension=OPENAI_CHATGPT_EXTENSION -2024/08/13 04:40:43.489795 INFO GetChatCompletionsStream recv for input text: [What's going on? ] sent sentence [ just here and ready to chat!] extension=OPENAI_CHATGPT_EXTENSION -2024/08/13 04:40:43.493444 INFO GetChatCompletionsStream recv for input text: [What's going on? ] sent sentence [ How about you?] extension=OPENAI_CHATGPT_EXTENSION -2024/08/13 04:40:43.495756 INFO GetChatCompletionsStream recv for input text: [What's going on? ] sent sentence [ What's on your mind?] extension=OPENAI_CHATGPT_EXTENSION -2024/08/13 04:40:43.496120 INFO GetChatCompletionsStream for input text: [What's going on? ] end of segment with sentence [] sent extension=OPENAI_CHATGPT_EXTENSION -``` -{% endcode %} - -When you see logs like this, it means the system is working correctly and logging each sentence you say.