-
Notifications
You must be signed in to change notification settings - Fork 371
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
36e4ea2
commit e1abe97
Showing
20 changed files
with
461 additions
and
174 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
# .github | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
# ISSUE\_TEMPLATE | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
--- | ||
name: Bug report | ||
about: Create a report to help us improve | ||
title: '' | ||
labels: '' | ||
assignees: '' | ||
|
||
--- | ||
|
||
**Describe the bug** | ||
A clear and concise description of what the bug is. | ||
|
||
**To Reproduce** | ||
Steps to reproduce the behavior: | ||
1. Go to '...' | ||
2. Click on '....' | ||
3. Scroll down to '....' | ||
4. See error | ||
|
||
**Expected behavior** | ||
A clear and concise description of what you expected to happen. | ||
|
||
**Screenshots** | ||
If applicable, add screenshots to help explain your problem. | ||
|
||
**Desktop (please complete the following information):** | ||
- OS: [e.g. iOS] | ||
- Browser [e.g. chrome, safari] | ||
- Version [e.g. 22] | ||
|
||
**Smartphone (please complete the following information):** | ||
- Device: [e.g. iPhone6] | ||
- OS: [e.g. iOS8.1] | ||
- Browser [e.g. stock browser, safari] | ||
- Version [e.g. 22] | ||
|
||
**Additional context** | ||
Add any other context about the problem here. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
--- | ||
name: Feature request | ||
about: Suggest an idea for this project | ||
title: '' | ||
labels: '' | ||
assignees: '' | ||
|
||
--- | ||
|
||
**Is your feature request related to a problem? Please describe.** | ||
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] | ||
|
||
**Describe the solution you'd like** | ||
A clear and concise description of what you want to happen. | ||
|
||
**Describe alternatives you've considered** | ||
A clear and concise description of any alternative solutions or features you've considered. | ||
|
||
**Additional context** | ||
Add any other context or screenshots about the feature request here. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
# Table of contents | ||
|
||
* [README](README.md) | ||
* [docs](docs/README.md) | ||
* [astra-architecture](docs/astra-architecture.md) | ||
* [Message Type and Name (简体中文)](docs/message-type-and-name-cn.md) | ||
* [Message Type and Name](docs/message-type-and-name.md) | ||
* [RTE Platform API Specification](docs/rte-api.md) | ||
* [Usage - RTE Schema (简体中文)](docs/rte-schema-cn.md) | ||
* [rte-schema](docs/rte-schema.md) | ||
* [code-of-conduct](docs/code-of-conduct/README.md) | ||
* [Contributor Covenant Code of Conduct](docs/code-of-conduct/code-of-conduct.md) | ||
* [Contribution Guidelines](docs/code-of-conduct/contributing.md) | ||
* [Security Policy](docs/code-of-conduct/security.md) | ||
* [readmes](docs/readmes/README.md) | ||
* [README-CN](docs/readmes/readme-cn.md) | ||
* [tutorials](docs/tutorials/README.md) | ||
* [Tutorial - 如何开发 C++ extension](docs/tutorials/extension-tutorial-cpp-cn.md) | ||
* [Overview](docs/tutorials/extension-tutorial-cpp.md) | ||
* [Tutorial - 如何开发 GO extension](docs/tutorials/extension-tutorial-go-cn.md) | ||
* [extension-tutorial-go](docs/tutorials/extension-tutorial-go.md) | ||
* [.github](.github/README.md) | ||
* [ISSUE\_TEMPLATE](.github/issue\_template/README.md) | ||
* [bug\_report](.github/issue\_template/bug\_report.md) | ||
* [feature\_request](.github/issue\_template/feature\_request.md) | ||
* [agents](agents/README.md) | ||
* [addon](agents/addon/README.md) | ||
* [extension](agents/addon/extension/README.md) | ||
* [azure\_tts](agents/addon/extension/azure\_tts.md) | ||
* [Amazon Bedrock LLM Extension](agents/addon/extension/bedrock\_llm\_python.md) | ||
* [openai\_chatgpt](agents/addon/extension/openai\_chatgpt.md) | ||
* [Amazon Polly TTS Extension](agents/addon/extension/polly\_tts.md) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
# agents | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
# addon | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
# extension | ||
|
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
## Amazon Bedrock LLM Extension | ||
|
||
### Configurations | ||
|
||
You can config this extension by providing following environments: | ||
|
||
| Env | Required | Default | Notes | | ||
| -- | -- | -- | -- | | ||
| AWS_REGION | No | us-east-1 | The Region of Amazon Bedrock service you want to use. | | ||
| AWS_ACCESS_KEY_ID | No | - | Access Key of your IAM User, make sure you've set proper permissions to [invoke Bedrock models](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html) and gain [models access](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html) in Bedrock. Will use default credentials provider if not provided. Check [document](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html). | | ||
| AWS_SECRET_ACCESS_KEY | No | - | Secret Key of your IAM User, make sure you've set proper permissions to [invoke Bedrock models](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html) and gain [models access](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html) in Bedrock. Will use default credentials provider if not provided. Check [document](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html). | | ||
| AWS_BEDROCK_MODEL | No | Claude 3.5(anthropic.claude-3-5-sonnet-20240620-v1:0) | Bedrock model id, check [docuement](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html#model-ids-arns). | |
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
# Amazon Polly TTS Extension | ||
|
||
## Configurations | ||
|
||
You can config this extension by providing following environments: | ||
|
||
| Env | Required | Default | Notes | | ||
| ----------------------------- | -------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | ||
| AWS\_TTS\_REGION | No | us-east-1 | The Region of Amazon Bedrock service you want to use. | | ||
| AWS\_TTS\_ACCESS\_KEY\_ID | No | - | Access Key of your IAM User, make sure you've set proper permissions to [synthesize speech](https://docs.aws.amazon.com/polly/latest/dg/security\_iam\_id-based-policy-examples.html#example-managed-policy-service-admin). Will use default credentials provider if not provided. Check [document](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html). | | ||
| AWS\_TTS\_SECRET\_ACCESS\_KEY | No | - | Secret Key of your IAM User, make sure you've set proper permissions to [synthesize speech](https://docs.aws.amazon.com/polly/latest/dg/security\_iam\_id-based-policy-examples.html#example-managed-policy-service-admin). Will use default credentials provider if not provided. Check [document](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html). | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
# docs | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,53 +1,65 @@ | ||
<h2>ASTRA Architecture</h2> | ||
# astra-architecture | ||
|
||
### ASTRA Architecture | ||
|
||
Now let's discuss what's under the hood. The ASTRA architecture is composed of various ASTRA extensions, developed in different programming languages. These extensions are interconnected using Graph, which describes their relationships and illustrates the flow of data. Furthermore, sharing and downloading extensions are simplified through the ASTRA Extension Store and the ASTRA Package Manager. | ||
|
||
![ASTRA](../images/image.png) | ||
![ASTRA](https://github.com/rte-design/ASTRA.ai/blob/main/images/image.png) | ||
|
||
\ | ||
|
||
<br> | ||
<h2>ASTRA Extension</h2> | ||
|
||
### ASTRA Extension | ||
|
||
An extension is the fundamental unit of composition within the ASTRA framework. Developers can create extensions in various programming languages and combine them to build diverse scenarios and applications. ASTRA emphasizes cross-language collaboration, allowing extensions written in different languages to work together seamlessly within the same application or service. | ||
|
||
For example, if an application requires real-time communication (RTC) features and advanced AI capabilities, a developer might choose to write RTC-related extensions in C++ for its performance advantages in processing audio and video data. Meanwhile, they could develop AI extensions in Python to leverage its extensive libraries and frameworks for data analysis and machine learning tasks. | ||
|
||
#### Supported Languages | ||
**Supported Languages** | ||
|
||
As of July 2024, we support extensions written in **C++**, **Golang** and **Python**. | ||
|
||
|
||
#### Extension Tutorials | ||
**Extension Tutorials** | ||
|
||
Here we have extension tutorials for C++ and Go. | ||
- [Extension Tutorial C++](./tutorials/extension-tutorial-cpp.md) | ||
- [Extension Tutorial Go](./tutorials/extension-tutorial-go.md) | ||
|
||
<br> | ||
<h2>Graph</h2> | ||
* [Extension Tutorial C++](tutorials/extension-tutorial-cpp.md) | ||
* [Extension Tutorial Go](tutorials/extension-tutorial-go.md) | ||
|
||
\ | ||
|
||
|
||
### Graph | ||
|
||
A Graph in ASTRA describes the data flow between extensions, orchestrating their interactions. For example, the text output from a speech-to-text (STT) extension might be directed to a large language model (LLM) extension. Essentially, a Graph defines which extensions are involved and the direction of data flow between them. Developers can customize this flow, directing outputs from one extension, such as an STT, into another, like an LLM. | ||
|
||
In ASTRA, there are four main types of data flow between extensions, they are **Command**, **Data**, **Image Frame** and **PCM Frame**. | ||
|
||
By specifying the direction of these data types in the Graph, developers can enable mutual invocation and unidirectional data flow between plugins. This is especially useful for PCM and image data types, simplifying audio and video processing. | ||
|
||
<br> | ||
<h2>ASTRA Agent App</h2> | ||
\ | ||
|
||
|
||
### ASTRA Agent App | ||
|
||
An ASTRA Agent App is a runnable server-side application that combines multiple Extensions following Graph rules to accomplish more sophisticated operations.An ASTRA Agent App is a robust, server-side application that executes complex operations by integrating multiple Extensions within a flexible framework defined by Graph rules. These Graph rules orchestrate the interplay between various Extensions, enabling the app to perform sophisticated tasks that go beyond the capabilities of individual components. | ||
|
||
By leveraging this architecture, an ASTRA Agent App can seamlessly manage and coordinate different functionalities, ensuring that each Extension interacts harmoniously with others. This design allows developers to create powerful and scalable applications capable of handling intricate workflows and data processing requirements. | ||
|
||
<br> | ||
<h2> ASTRA Extension Store</h2> | ||
\ | ||
|
||
|
||
### ASTRA Extension Store | ||
|
||
The ASTRA Store is a centralized platform designed to foster collaboration and innovation among developers by providing a space where they can share their extensions. This allows developers to contribute to the community, showcase their work, and receive feedback from peers, enhancing the overall quality and functionality of the ASTRA ecosystem. | ||
|
||
In addition to sharing their own extensions, developers can also access a wide array of extensions created by others. This extensive library of extensions makes it easier to find tools and functionalities that can be integrated into their own projects, accelerating development and promoting best practices within the community. The ASTRA Store thus serves as a valuable resource for both novice and experienced developers looking to expand their capabilities and leverage the collective expertise of the community. | ||
|
||
<br> | ||
<h2> ASTRA Package Manager</h2> | ||
\ | ||
|
||
|
||
### ASTRA Package Manager | ||
|
||
The ASTRA Package Manager streamlines the entire process of handling ASTRA extensions, making it easy to upload, share, download, and install them. It significantly simplifies the workflow by allowing extensions to specify their dependencies on other extensions and the environment. This ensures that all necessary components are automatically managed and installed, reducing the potential for errors and conflicts. | ||
|
||
By automatically managing these dependencies, the ASTRA Package Manager makes the installation and release of extensions extremely convenient and intuitive. This tool not only saves time but also enhances the user experience by ensuring that every extension works seamlessly within the larger ecosystem. This level of automation and ease of use encourages the development and distribution of more robust and complex extensions, further enriching the ASTRA framework. | ||
By automatically managing these dependencies, the ASTRA Package Manager makes the installation and release of extensions extremely convenient and intuitive. This tool not only saves time but also enhances the user experience by ensuring that every extension works seamlessly within the larger ecosystem. This level of automation and ease of use encourages the development and distribution of more robust and complex extensions, further enriching the ASTRA framework. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
# code-of-conduct | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
# readmes | ||
|
Oops, something went wrong.