Ecoute is a live transcription tool that provides real-time transcripts for both the user's microphone input (You) and the user's speakers output (Speaker) in a textbox. It also generates a suggested response using OpenAI's GPT-3.5 for the user to say based on the live transcription of the conversation.
This fork supports macOS. See config details below
demo.mp4
Ecoute is designed to help users in their conversations by providing live transcriptions and generating contextually relevant responses. By leveraging the power of OpenAI's GPT-3.5, Ecoute aims to make communication more efficient and enjoyable.
Follow these steps to set up and run Ecoute on your local machine.
- Python >=3.8.0
- An OpenAI API key that can access OpenAI API (set up a paid account OpenAI account)
- FFmpeg (check below for installation instructions)
If FFmpeg is not installed in your system, you can follow the steps below to install it.
First, you need to install Chocolatey, a package manager for Windows. Open your PowerShell as Administrator and run the following command:
Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))
Once Chocolatey is installed, you can install FFmpeg by running the following command in your PowerShell:
choco install ffmpeg
Please ensure that you run these commands in a PowerShell window with administrator privileges. If you face any issues during the installation, you can visit the official Chocolatey and FFmpeg websites for troubleshooting.
Dependencies and tools
- XCode Command Line Tools
brew install portaudio python-tk ffmpeg blackhole-2ch
BlackHole configuration
Setup "Multi-Ouput Device" and set it as default sound output device for your macOS. Guidelines are available here
Configuring device names
Speakers audio on macOS will be recorded from the virtual "BlackHole 2ch" microphone. Your and BlackHole microphone device names could be adjusted via HUMAN_MIC_NAME and BLACKHOLE_MIC_NAME vars in the AudioRecorder.py.
Run python fetch_sr_data.py to get speaker and microphone devices list and their indices.
-
Clone the repository:
git clone https://github.com/SevaSk/ecoute -
Navigate to the
ecoutefolder:cd ecoute -
Install the required packages:
pip install -r requirements.txt -
Create a
keys.pyfile in the ecoute directory and add your OpenAI API key:-
Option 1: You can utilize a command on your command prompt. Run the following command, ensuring to replace "API KEY" with your actual OpenAI API key:
python -c "with open('keys.py', 'w', encoding='utf-8') as f: f.write('OPENAI_API_KEY=\"API KEY\"')" -
Option 2: You can create the keys.py file manually. Open up your text editor of choice and enter the following content:
OPENAI_API_KEY="API KEY"Replace "API KEY" with your actual OpenAI API key. Save this file as keys.py within the ecoute directory.
-
Run the main script:
python main.py
For a more better and faster version that also works with most languages, use:
python main.py --api
Upon initiation, Ecoute will begin transcribing your microphone input and speaker output in real-time, generating a suggested response based on the conversation. Please note that it might take a few seconds for the system to warm up before the transcription becomes real-time.
The --api flag will use the whisper api for transcriptions. This significantly enhances transcription speed and accuracy, and it works in most languages (rather than just English without the flag). It's expected to become the default option in future releases. However, keep in mind that using the Whisper API will consume more OpenAI credits than using the local model. This increased cost is attributed to the advanced features and capabilities that the Whisper API provides. Despite the additional expense, the substantial improvements in speed and transcription accuracy may make it a worthwhile investment for your use case.
While Ecoute provides real-time transcription and response suggestions, there are several known limitations to its functionality that you should be aware of:
Default Mic and Speaker: not applicable for macOS Ecoute is currently configured to listen only to the default microphone and speaker set in your system. It will not detect sound from other devices or systems. If you wish to use a different mic or speaker, you will need to set it as your default device in your system settings.
Whisper Model: If the --api flag is not used, we utilize the 'tiny' version of the Whisper ASR model, due to its low resource consumption and fast response times. However, this model may not be as accurate as the larger models in transcribing certain types of speech, including accents or uncommon words.
Language: If you are not using the --api flag the Whisper model used in Ecoute is set to English. As a result, it may not accurately transcribe non-English languages or dialects. We are actively working to add multi-language support to future versions of the program.
This project is licensed under the MIT License - see the LICENSE file for details.
Contributions are welcome! Feel free to open issues or submit pull requests to improve Ecoute.