Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Readme update #88

Merged
merged 8 commits into from
Jul 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# next.js
/.next/
/out/

*.log
addon/extension_group/
agoradns.dat
Expand Down
84 changes: 50 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,11 @@
<a href="./README.md"><img alt="README in English" src="https://img.shields.io/badge/English-lightgrey"></a>
<a href="./README-CN.md"><img alt="简体中文" src="https://img.shields.io/badge/简体中文-lightgrey"></a>

[Lightning Fast](https://github.com/rte-design/ASTRA.ai/blob/main/astra-service.md#astra-service)
[Lightning Fast](https://github.com/rte-design/ASTRA.ai/blob/main/astra-architecture.md#astra-architecture)
<span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
[Multimodal Interactive](https://github.com/rte-design/ASTRA.ai/blob/main/astra-service.md#astra-extension)
[Multimodal Interactive](https://github.com/rte-design/ASTRA.ai/blob/main/astra-architecture.md#astra-extension)
<span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
[Highly Customizable](https://github.com/rte-design/ASTRA.ai/blob/main/astra-service.md#astra-extension)
[Highly Customizable](https://github.com/rte-design/ASTRA.ai/blob/main/astra-architecture.md#astra-extension)

</div>

Expand All @@ -39,7 +39,7 @@ ASTRA is a versatile platform capable of building a wide range of agents. Here,

<h3>Run Voice Agent Locally</h3>

Of course, you are more than welcome to run the example voice agent locally. We have a Docker image ready for you to build and run the agent on both macOS and Windows.
Of course, you are more than welcome to run the showcased voice agent locally. We have a Docker image ready for you to build and run the agent on both macOS and Windows.

To start, make sure you have:

Expand All @@ -49,7 +49,7 @@ To start, make sure you have:
- [Docker](https://www.docker.com/)

```bash
# run the pre-built agent image
# Run the pre-built agent image
docker run --restart=always -itd -p 8080:8080 \
-v /tmp:/tmp \
-e AGORA_APP_ID=<your_agora_appid> \
Expand All @@ -63,6 +63,8 @@ docker run --restart=always -itd -p 8080:8080 \
agoraio/astra_agents_server:latest
```

This should start an agent server running on port 8080.

#### Mac with Apple Silicon

You will need to uncheck "Use Rosetta for x86_64/amd64 emulation on apple silicon" option for Docker if you are on Apple Silicon.
Expand All @@ -73,41 +75,38 @@ You will need to uncheck "Use Rosetta for x86_64/amd64 emulation on apple silico

</div>

This should start an agent server running on port 8080.

<h3>Connect to Your Agent</h3>

You can use the playground project to test with the server you just started.
You can use the showcase voice agent, in `/playground` folder, to test with the server you just started.

The Playground project is built on NextJS 14, hence it needs Node 18+.
The project is built on NextJS 14, hence it needs Node 18 or later.

```bash
# set up an .env file
# Set up an .env file
cp ./playground/.env.example ./playground/.env
cd playground

# install npm dependencies & start
# Install npm dependencies & start
npm i && npm run dev
```

🎉 Congratulations! You now have our ASTRA powered voice agent running locally.
🎉 Congratulations! You now have a ASTRA powered voice agent running locally.

<br />
<br>
<h2>Agent Customization</h2>

Our voice agent is a great place for you to start with, it uses following Extensions:
To explore further, the ASTRA voice agent is an excellent starting point. It incorporates the following extensions, some of which will be interchangeable in the near future. Feel free to choose the ones that best suit your needs and maximize ASTRA’s capabilities.

- _agora_rtc_ / [Agora](https://docs.agora.io/en) for RTC transport + VAD + Azure speech-to-text (STT)
- _azure_tts_ / [Azure](https://azure.microsoft.com/en-us/products/ai-services/text-to-speech) for text-to-speech (TTS)
- _openai_chatgpt_ / [OpenAI](https://openai.com/index/openai-api/) for LLM
- _chat_transcriber_ / A utility ext to forward chat logs into channel
- _interrupt_detector_ / A utility ext to help interrupt agent
| Extension | Feature | Description |
|---------------------|-----------------|-------------------------------------------------------------------------------------------------------------|
| openai_chatgpt | LLM | [ GPT-4o ](https://platform.openai.com/docs/models/gpt-4o), [ GPT-4 Turbo ](https://platform.openai.com/docs/models/gpt-4-turbo-and-gpt-4), [ GPT-3.5 Turbo ](https://platform.openai.com/docs/models/gpt-3-5-turbo) |
| elevenlabs_tts | Text-to-speech | [ElevanLabs text to speech](https://elevenlabs.io/) converts text to audio |
| azure_tts | Text-to-speech | [Azure text to speech](https://azure.microsoft.com/en-us/products/ai-services/text-to-speech) converts text to audio |
| azure_stt | Speech-to-text | [Azure speech to text](https://azure.microsoft.com/en-us/products/ai-services/speech-to-text) converts audio to text |
| chat_transcriber | Transcriber | A utility ext to forward chat logs into channel |
| agora_rtc | Transporter | A low latency transporter powered by agora_rtc |
| interrupt_detector | Interrupter | A utility ext to help interrupt agent |

<div align="center">

<image alt="ASTRA" src="./images/image-2.png">

</div>

<h3>Customize Agent</h3>

Expand All @@ -116,41 +115,60 @@ You might want to add more flavors to make the agent better suited to your needs
You need to prepare the proper `manifest.json` file first.

```bash
# rename manifest example
# Rename manifest example
cp ./agents/manifest.json.example ./agents/manifest.json

# pull the docker image with dev tools and mount your current folder as workspace
# Pull the docker image with dev tools and mount your current folder as workspace
docker run -itd -v $(pwd):/app -w /app -p 8080:8080 --name astra_agents_dev agoraio/astra_agents_build

# enter docker image
# Enter docker image
docker exec -it astra_agents_dev bash

# build agent
# Build agent
make build
```

This code generates an agent executable. To customize your prompts and OpenAI parameters, modify the source code in agents/addon/extension/openai_chatgpt/openai_chatgpt.go.
The above code generates an agent executable. To customize your prompts and OpenAI parameters, modify the source code in `agents/addon/extension/openai_chatgpt/openai_chatgpt.go`.

<h3>Start Server</h3>

Once you have made the necessary changes, you can use the following commands to start a server. You can then test it out using the ASTRA voice agent playground as we did in previous steps.
Once you have made the necessary changes, you can use the following commands to start a server. You can then test it out using the ASTRA voice agent from the showcase.

```bash

# Agora App ID and Agora App Certificate
export AGORA_APP_ID=<your_agora_appid>
export AGORA_APP_CERTIFICATE=<your_agora_app_certificate>

# OpenAI API key
export OPENAI_API_KEY=<your_openai_api_key>

# Azure STT key and region
export AZURE_STT_KEY=<your_azure_stt_key>
export AZURE_STT_REGION=<your_azure_stt_region>
export OPENAI_API_KEY=<your_openai_api_key>

# Here are two TTS options, either one will work
# Make sure to comment out the one you don't use

# 1. using Azure
export TTS_VENDOR_CHINESE=azure
export AZURE_TTS_KEY=<your_azure_tts_key>
export AZURE_TTS_REGION=<your_azure_tts_region>

# 2. using ElevenLabs
export TTS_VENDOR_ENGLISH=elevenlabs
export ELEVENLABS_TTS_KEY=<your_elevanlabs_tts_key>

# agent is ready to start on port 8080

make run-server
```

🎉 Congratulations! You have created your first personalized voice agent.

To understand how the ASTRA service works under the hood, please refer to the [ ASTRA service documentation ](astra-service.md).
<h3>Discover More</h3>

Now that you’ve created your first AI agent, the creativity doesn’t stop here. To develop more amazing agents, you’ll need an advanced understanding of how the ASTRA works under the hood. Please refer to the [ ASTRA architecture documentation ](astra-architecture.md).

<br>
<h2>Stay Tuned</h2>
Expand All @@ -160,8 +178,6 @@ Star our repository and get instant notifications for all new releases!

![ASTRA star us gif](https://github.com/rte-design/ASTRA.ai/raw/main/images/star-the-repo-confetti-higher-quality.gif)

<br>

<br>
<h2>Join Community</h2>

Expand Down
4 changes: 2 additions & 2 deletions astra-service.md → astra-architecture.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<h2>ASTRA Service</h2>
<h2>ASTRA Architecture</h2>

Now let's discuss what's under the hood. The ASTRA Service is composed of various ASTRA extensions, developed in different programming languages. These extensions are interconnected using Graph, which describes their relationships and illustrates the flow of data. Furthermore, sharing and downloading extensions are simplified through the ASTRA Extension Store and the ASTRA Package Manager.
Now let's discuss what's under the hood. The ASTRA architecture is composed of various ASTRA extensions, developed in different programming languages. These extensions are interconnected using Graph, which describes their relationships and illustrates the flow of data. Furthermore, sharing and downloading extensions are simplified through the ASTRA Extension Store and the ASTRA Package Manager.

<div align="center">

Expand Down