Voice Assistant-Add support for the espressif esp32-korvo-v1.1 #2430
Comments
|
Searching, got me intrigued... The Korvo appears to be It looks like it may be using the same codebase as the what Espressif calls ESP-BOX (ie the ESP32-S3-BOX)... which puts it under the scope of #2239 - doesn't it? |
|
hope someone could study how to add it into ESPHOME. it's it include a ADC es7210 and es8311CODEC and LED control chip. |
|
Got one on order, will start playing with it as soon as I receive it. |
|
I managed to get an ESP32-s3-Korvo-1 wooing with the setup posted https://github.com/joey-90/ESP32-S3-Korvo-1---Voice-Assistant/blob/main/voiceassistant.yaml I can't get wake word detection working at the moment though |
Thanks @joey-90 for this. I copied the contents of the yaml from your repo and tried to compile/load for my board but I get these msgs when compiling. The firmware loads into the board but none of the i2s components load/work. Have you seen these warnings/errors when compiling? I am running ESPHome 2023.10.3. |
|
I have the same compilation warnings and the same (non VAD working) behaviour with my Korvo-2. I used the S3-box yaml. Same warnings during compilation, looking good during the device esphome startup log (as far as I know) but also no wakeword detection. As per Jessie it should work very similar to the S3-BOX though ... https://discord.com/channels/429907082951524364/1163574334472863815/1166079003950588006 |
|
I've updated the code posted in the repository as I have now got wake word detection working. In team of compile errors, I did get some initially but was mostly caused by fat fingering repository names and not having the correct pin layout. I think the Korvo 2 and the non S3 Korvo have different Pin outs. The key to getting it working mostly for me was using the esp32-s3-devkitc-1 board variant. Have you got the code for this posted anywhere? |
Thanks, I'll dbl chk my board version, gpio pins. |
|
I managed to get this working... Heres' the gpio pinouts I used: The microphone, wakeword, LED all work fine. A big thank you to @joey-90 for the yaml. |
|
A-ha. Is it the WROOM-32-based module here? https://www.aliexpress.com/item/1005002803964499.html For about $30, it's a steal if this can actually act as a voice assistant. |
Yes, that's the one I am using. |
|
That's the same board I ordered. Thanks to you guys for the work, I'll try it out as soon as I receive it. |
Can you post somewhere the full YAML? I just ordered two of these, one for my test bench and one to create a nice setup once I can get a cool 3D printed case. |
This is the full yaml I used. Please note I didn't manage to successfully test the speaker output. I am not sure if it was the speaker I was using or the yaml cfg. substitutions:
friendly_name: esp32-voice-3
esphome:
name: esp32-voice-3
platformio_options:
board_build.flash_mode: dio
on_boot:
- priority: -100
then:
- wait_until: api.connected
- delay: 1s
- if:
condition:
switch.is_on: use_wake_word
then:
- voice_assistant.start_continuous:
esp32:
board: esp-wrover-kit
framework:
#type: esp-idf
type: arduino
version: recommended
external_components:
- source: github://rpatel3001/esphome@es8311
components: [ es8311 ]
- source: github://rpatel3001/esphome@es7210
components: [ es7210 ]
- source: github://pr#5230
components:
- esp_adf
# Enable logging
logger:
# Enable Home Assistant API
api:
encryption:
key: <REDACTED>
ota:
password: <REDACTED>
wifi:
ssid: <REDACTED>
password: <REDACTED>
use_address: <REDACTED>
i2c:
sda: GPIO19 #GPIO1
scl: GPIO32 #GPIO2
scan: true
frequency: 400kHz
es8311:
address: 0x18
es7210:
address: 0x40
output:
- platform: gpio
id: pa_ctrl
pin: GPIO12 #GPIO38
i2s_audio:
- id: codec
i2s_lrclk_pin: GPIO22 #GPIO41 #ws
i2s_bclk_pin: GPIO25 #GPIO40 #clk
i2s_mclk_pin: GPIO0 #GPIO42
- id: mic_adc
i2s_lrclk_pin: GPIO26 #GPIO9 #ws
i2s_bclk_pin: GPIO27 #GPIO10 #clk
i2s_mclk_pin: GPIO0 #GPIO20
speaker:
- platform: i2s_audio
id: external_speaker
dac_type: external
i2s_audio_id: codec
i2s_dout_pin: GPIO13 #GPIO39
mode: mono
microphone:
- platform: i2s_audio
id: external_mic
adc_type: external
i2s_audio_id: mic_adc
i2s_din_pin: GPIO36 #GPIO11
pdm: false
voice_assistant:
id: voice_asst
microphone: external_mic
speaker: external_speaker
noise_suppression_level: 2
auto_gain: 15dBFS
volume_multiplier: 0.5
use_wake_word: false
on_listening:
- light.turn_on:
id: led_ring
blue: 100%
red: 0%
green: 0%
brightness: 100%
effect: wakeword
on_tts_start:
- light.turn_on:
id: led_ring
blue: 0%
red: 0%
green: 100%
brightness: 50%
effect: pulse
on_end:
- delay: 100ms
- wait_until:
not:
speaker.is_playing:
- script.execute: reset_led
on_error:
- light.turn_on:
id: led_ring
blue: 0%
red: 100%
green: 0%
brightness: 100%
effect: none
- delay: 1s
- script.execute: reset_led
- script.wait: reset_led
- lambda: |-
if (code == "wake-provider-missing" || code == "wake-engine-missing") {
id(use_wake_word).turn_off();
}
script:
- id: reset_led
then:
- if:
condition:
switch.is_on: use_wake_word
then:
- light.turn_on:
id: led_ring
blue: 30%
red: 0%
green: 0%
brightness: 25%
effect: none
else:
- light.turn_off: led_ring
switch:
- platform: template
name: Use wake word
id: use_wake_word
optimistic: true
restore_mode: RESTORE_DEFAULT_ON
entity_category: config
on_turn_on:
- lambda: id(voice_asst).set_use_wake_word(true);
- if:
condition:
not:
- voice_assistant.is_running
then:
- voice_assistant.start_continuous
- script.execute: reset_led
on_turn_off:
- voice_assistant.stop
- script.execute: reset_led
light:
- platform: esp32_rmt_led_strip
id: led_ring
name: "${friendly_name} Light"
pin: GPIO33 #GPIO19
num_leds: 12
rmt_channel: 0
rgb_order: GRB
chipset: ws2812
default_transition_length: 0s
effects:
- pulse:
name: "Pulse"
transition_length: 0.5s
update_interval: 0.5s
- addressable_twinkle:
name: "Working"
twinkle_probability: 5%
progress_interval: 4ms
- addressable_color_wipe:
name: "Wakeword"
colors:
- red: 0%
green: 50%
blue: 0%
num_leds: 12
add_led_interval: 40ms
reverse: false
binary_sensor:
- platform: template
name: "${friendly_name} Volume Up"
id: btn_volume_up
- platform: template
name: "${friendly_name} Volume Down"
id: btn_volume_down
- platform: template
name: "${friendly_name} Set"
id: btn_set
- platform: template
name: "${friendly_name} Play"
id: btn_play
- platform: template
name: "${friendly_name} Mode"
id: btn_mode
- platform: template
name: "${friendly_name} Record"
id: btn_record
on_press:
- output.turn_on: pa_ctrl
- voice_assistant.start:
- light.turn_on:
id: led_ring
brightness: 100%
effect: "Wakeword"
on_release:
- voice_assistant.stop:
- output.turn_off: pa_ctrl
- light.turn_off:
id: led_ring
sensor:
- id: button_adc
platform: adc
internal: true
pin: 39 #8
attenuation: 11db
update_interval: 15ms
filters:
- median:
window_size: 5
send_every: 5
send_first_at: 1
- delta: 0.1
on_value_range:
- below: 0.55
then:
- binary_sensor.template.publish:
id: btn_volume_up
state: ON
- above: 0.65
below: 0.92
then:
- binary_sensor.template.publish:
id: btn_volume_down
state: ON
- above: 1.02
below: 1.33
then:
- binary_sensor.template.publish:
id: btn_set
state: ON
- above: 1.43
below: 1.77
then:
- binary_sensor.template.publish:
id: btn_play
state: ON
- above: 1.87
below: 2.15
then:
- binary_sensor.template.publish:
id: btn_mode
state: ON
- above: 2.25
below: 2.56
then:
- binary_sensor.template.publish:
id: btn_record
state: ON
- above: 2.8
then:
- binary_sensor.template.publish:
id: btn_volume_up
state: OFF
- binary_sensor.template.publish:
id: btn_volume_down
state: OFF
- binary_sensor.template.publish:
id: btn_set
state: OFF
- binary_sensor.template.publish:
id: btn_play
state: OFF
- binary_sensor.template.publish:
id: btn_mode
state: OFF
- binary_sensor.template.publish:
id: btn_record
state: OFF
``` |
|
Great input folks! Just received my device and tested it out. I have a classic wired apple headphone connected to the jack. |
|
Has anyone tried putting an enclosure on this yet? I'm also curious about the dimensions. The only spec I found said it's 88mm wide but no mention of the height - especially since it appears to be 2 boards strapped together... pretty hard to find 2 prefabricated plastic enclosures... Is 3D printing the best option? |
|
Just received my board today. If I get a chance this weekend to set it up, I'll work on designing a case that I have mind. |
|
Just tried to program the board. When I connect the USB, it does not get recognized as a serial port, when I connect a FTDI board to the pads on the PCB, it does not recognize the type of ESP chip. |
Initially I thought I had the same issue with my board (not detected), but the issue was that I needed to connect power to the board (via the usb power connector) and connect your PC via the UART usb port on the board. |
|
Never mind, thanks for the help. |
|
@joey-90 thanks for your config! It works great except the speaker. Did you got that to work? On mine there is no sound (I'm using the headphone jack) |
|
I couldn't get the speaker to work either. I verified the I/O pins against the schematic, and they seem to match correctly. |
|
Apologies for the delay in getting back to everyone, its been a busy week or two with work etc. I also can't get the speaker output to work, the pin out is definitely correct. The same pin config is used in the demo firmware from Espressif. I get similar error messages to what you are seeing. I've got some updated yaml config to upload where I've been playing with some of the values for the Voice assistant etc. |
|
Great! thank you, I would like to get this working. |
|
I received mine today and can't get Assistant to work. When pressing the record button and releasing it nothing happens and I also don't receive any events in Home Asssitant. The Debug Page of my Voice Assistant says: Besides of that I can see the event of the button press and the LED state change in Home Assistant from the device. I used the same config as @asve99 Anyone with an idea why this is not working for me? |
The buttons aren't tied to any functions at the moment, you have to leverage a wake word from my personal experience. |
|
@pascalmtts see @joey-90 example on how to use the buttons. In his example voice is also working fine, but no sound output yet. I am currently trying to add the Korvo1 board to ESP-ADF, which in turn should make it easy to get it into ESPHome. |
|
The button record is tied to a function in the template above: As I did not get the wake word to work I tried using the record button, as it should start and stop the voice assistant.
That's great! I don't know if you also have the Korvo1.1 on the list because I think these are 2 different boards. The Korvo1.1 is much cheaper then ESP32-S3-Korvo-1 Edit: Here is a log output from ESPHome when pressing, holding and then releasing the rec button: |
|
Alright I got it working. Problem was in my Assist configuration - after selecting Piper as TTS it is working now. Audio is also not working in my case, but I think we will find a solution for this in the future |
|
I've uploaded new YAML config having made some tweaks. These are:
Still no success with the sound output, but there is something to try. Will post over the weekend when I've had a chance to play. Things to add:
Please comment if there any suggestions and feel free to add pull requests if you see any errors or can add better code. |
|
Are you keeping an eye on all the related experimental components in the esphome organization's new voice-kit repository? https://github.com/esphome/voice-kit https://github.com/esphome/voice-kit/tree/dev/esphome/components https://github.com/esphome/voice-kit/pulls?q=is%3Apr+ https://github.com/esphome/voice-kit/issues?q=is%3Aissue+ |
Interesting links, I'll keep an eye on this. |
Hello @dwitgen |
|
Unfortunately, I am not maintaining it. |
|
So you just tossed it, breaking anyone who had been using it? Did anyone else grab a fork of it? |
The korvo 1 never did work correctly. If you let me know what repo/s you were using I can load them back up for a day or so and you can clone them. I was only doing some testing and put them up if anyone wanted to clone them. In hindsight I probably should not have done that as I really did not have the means to properly put something out for public use. |
|
Well, specifically, I think the problem people have with it is getting the speaker working. The matrix microphone is pretty straightforward. Your configuration is the only one I've seen, anywhere, where someone got sound output working. I don't know if its something in your esphome fork adding adf support, or something in your board definition. But other people's ones -- including for the 8311 -- seem to universally result in no sound output. So there's something you were doing in your branch to initialize the 8311, or how you configured the i2s to talk to it, that no one else has figured out. That could potentially be ported to someone else's components, but it'd take knowing what you'd done differently. So I guess the esphome fork you had is probably the most useful one, but getting both where people can grab a copy would give the best odds of working out what you did differently. Given how ideal a platform the Korvo-1 is, I think that's why people are so interested. Short of whatever the Nabu Casa people release, it's the best platform for a voice assistant. |
|
My Korvo 1.1 broke my heart. Got it to work one night at least to listen to voice commands. Went to bed excited and tried to show my wife the next day and it made a fool out of me. Spent too many nights on that thing. It's a paperweight somewhere... for now. That's a danger of this hobby. Too many hardware manufacturers release untested gear on the community and if we make it work, then it's a success... if not, the makers move onto something else. But didn't the ESP32-S2 and S3 take 3-4 years to finally reach mainstream support in Tasmota and ESPHome? The Korvos are just getting started... |
Well, the strange thing is there's not really any hardware in it that you don't find in other devices that work fine. So it's really some combination of the board definition and difference in the esp-adf that is doing it. I have other ESP32-S3 devices using the same or similar DAC and ADC chips, and they work fine. I suspect I'm going to end up standardizing on the Waveshare ESP32-S3-LCD-1.85, which works flawlessly for me, but doesn't have the microphone range. But I'd rather standardize on the Korvo-1 because of the better mic setup. In testing, my work flawlessly from anywhere in a fairly large room at "normal" talking levels, whereas I feel like I have to speak loudly to the Waveshare board. But really, any of the Xtensa based ESP32s should, broadly speaking, work. The shift to Risc-V for the newer ESP32 chips, though, makes me concerned about ongoing support. |
did you manage to get the Korvo-2 working? im having no luck at all, flashes and responds to wireless logs ok but speaker and mic dont work, i suspect the pin mapping is wrong |
|
I was using and having very good luck with it. I just had to reboot the first power on. I wish you would restore the depository for people. I just went to update it tonight and got the same error about the directory. |
|
@MechlingBurgh, does that mean you still have the last pull you did? I know a lot of people use ESPHome without a lot of technical experience, but do you know enough to change the git remote and push it up to another place on github, if so? A git remote set-url and a push is all it would take. Someone with a local repo clone can get it back online even if @dwitgen doesn't pop back up. Being able to see the patches made to esphome would be handy for anyone else figuring out why this branch worked and all the other ones people have done haven't been quite right. Even a copy of the code would be good, but a copy of the whole repo would keep the change history intact. |
|
So, the repos are back up, but I cannot maintain them if someone wants to pull them. I did not find the Korvo v1 to work very well. The main changes were to add support to the Korvo v1 board and add the board/pin configuration files (which is the test_boards repo) so that it would work with the ESP_ADF. There were some changes made to add volume and pa functions directly to the speaker component but again all this was just some testing on my part and cannot say this is the correct way for this to be done. Not sure how long they will be up, probably not past the first time they have an issue due to changes in ESPHome/ESP-IDF/ESP-ADF. Did not mean to cause any inconvenience. |
|
Thanks for that! I've forked them, and will leave them up, for anyone who wants them. Hopefully it helps work out what the limitations were for other ADF ports, so we can get support working long-term. https://github.com/dotorg/korvo_1_esphome I have no plans to touch them, so any configs currently working with the old repos ought to keep working with these. If I end up making any incompatible changes, it'll be via a different component. But, honestly, I suspect the issue with the more common ADF ports not working is just an initialization one, and I believe it'll be possible to do that in straight YAML code, which will be more stable relative to whatever ADF port people are using. |
Ya I will copy them to my local drive and change it in the yaml, thank you. I don't know a lot but I'm learning as I do more a lot of trail and error. |
|
@dwitgen thank you for your sources. ---
substitutions:
name: esp32-korvo1-mww
friendly_name: Korvo 1 microWakeWord
voice_assist_idle_phase_id: "1"
voice_assist_listening_phase_id: "2"
voice_assist_thinking_phase_id: "3"
voice_assist_replying_phase_id: "4"
voice_assist_not_ready_phase_id: "10"
voice_assist_error_phase_id: "11"
voice_assist_muted_phase_id: "12"
micro_wake_word_model: okay_nabu
esphome:
name: "${name}"
friendly_name: "${friendly_name}"
min_version: 2024.5.5
platformio_options:
board_build.flash_mode: dio
on_boot:
- priority: -100
then:
- light.turn_on:
id: led_ring
blue: 0%
red: 100%
green: 0%
effect: Fast Pulse
- delay: 1s
- wait_until:
condition:
wifi.connected:
- light.turn_on:
id: led_ring
blue: 0%
red: 100%
green: 50%
effect: Slow Pulse
- wait_until:
condition: api.connected
- lambda: id(init_in_progress) = false;
- lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
- script.execute: reset_led
esp32:
board: esp-wrover-kit
flash_size: 16MB
framework:
type: esp-idf
version: recommended
sdkconfig_options:
CONFIG_IDF_TARGET_ESP32: y
CONFIG_ESPTOOLPY_FLASHMODE_QIO: y
CONFIG_ESPTOOLPY_FLASHFREQ_80M: y
CONFIG_ESPTOOLPY_FLASHSIZE_16MB: y
CONFIG_ESP32S3_DATA_CACHE_64KB: y
CONFIG_ESP32S3_DATA_CACHE_LINE_64B: y
CONFIG_PARTITION_TABLE_CUSTOM: y
CONFIG_PARTITION_TABLE_CUSTOM_FILENAME: "default_16MB.csv"
CONFIG_PARTITION_TABLE_FILENAME: "default_16MB.csv"
CONFIG_PARTITION_TABLE_OFFSET: "0x8000"
CONFIG_ESP32_DEFAULT_CPU_FREQ_240: y
CONFIG_ESP32_SPIRAM_SUPPORT: y
CONFIG_SPIRAM_SPEED_80M: y
CONFIG_SPIRAM_SIZE: "4194304" # 4MB
CONFIG_SPIRAM_USE: y
CONFIG_ESP_SYSTEM_PANIC_SILENT_REBOOT: y
CONFIG_I2S_ENABLE_DEBUG_LOG: y
CONFIG_AUDIO_BOARD_CUSTOM: y
components:
- name: esp32_korvo1_board #esp32_s3_korvo1_board for the s3 variant and really you should be able to name this anything
source: github://chymaslik/test_boards@main
refresh: 0s
external_components:
- source: github://chymaslik/esphome@main #pr#5230
components: [esp_adf]
refresh: 0s
# Enable logging
logger:
logs:
component: ERROR
ota:
- platform: esphome
password: "e4125183bb6d45bf43ab81cfd6160d23"
# Enable Home Assistant API
api:
encryption:
key: "mAP0fLzUWLw7MJfpOY3MquhMm6lEvbT9uzMPDWVyfgE="
services:
- service: volume_up
then:
- lambda: |-
if (id(speaker_volume) < 100) {
id(external_speaker).volume_up();
}
- service: volume_down
then:
- lambda: |-
if (id(speaker_volume) > 0) {
id(external_speaker).volume_down();
}
# text_sensor:
# - platform: wifi_info
# ip_address:
# name: "${friendly_name} IP Address"
# time:
# platform: homeassistant
# id: homeassistant_time
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
manual_ip:
static_ip: 192.168.44.243
subnet: 255.255.255.0
gateway: 192.168.44.1
# # Enable fallback hotspot (captive portal) in case wifi connection fails
# ap:
# ssid: "Esp32-Korvo-1"
# password: !secret wifi_password
# captive_portal:
output:
- platform: gpio
id: pa_ctrl
pin:
number: GPIO12
ignore_strapping_warning: true
esp_adf:
board: esp32korvo1 #esp32s3korvo1 for s3 variant
speaker:
- platform: esp_adf
id: external_speaker
microphone:
- platform: esp_adf
id: external_mic
micro_wake_word:
models: ${micro_wake_word_model} #okay_nabu
on_wake_word_detected:
then:
- voice_assistant.start:
wake_word: !lambda return wake_word;
voice_assistant:
id: voice_asst
microphone: external_mic
speaker: external_speaker
noise_suppression_level: 2
auto_gain: 31dBFS
volume_multiplier: 2.0
on_listening:
- lambda: id(voice_assistant_phase) = ${voice_assist_listening_phase_id};
- script.execute: reset_led
on_stt_vad_end:
- lambda: id(voice_assistant_phase) = ${voice_assist_thinking_phase_id};
- script.execute: reset_led
on_tts_start:
- light.turn_on:
id: led_ring
blue: 0%
red: 100%
green: 100%
brightness: 60%
effect: Working
on_stt_end:
- homeassistant.service:
service: media_player.play_media
data:
entity_id: media_player.ke_ting
media_content_id: !lambda return x;
media_content_type: music
announce: "true"
on_tts_stream_start:
- output.turn_on: pa_ctrl
- delay: 100ms
- lambda: id(voice_assistant_phase) = ${voice_assist_replying_phase_id};
- script.execute: reset_led
on_end:
- delay: 100ms
- wait_until:
not:
speaker.is_playing:
- lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
- script.execute: reset_led
- if:
condition:
and:
- switch.is_off: mute
- lambda: return id(wake_word_engine_location).state == "On device";
then:
- wait_until:
not:
voice_assistant.is_running:
- micro_wake_word.start:
on_error:
- if:
condition:
lambda: return !id(init_in_progress);
then:
- lambda: id(voice_assistant_phase) = ${voice_assist_error_phase_id};
- script.execute: reset_led
- delay: 2s
- if:
condition:
switch.is_off: mute
then:
- lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
else:
- lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id};
- script.execute: reset_led
on_client_connected:
- if:
condition:
switch.is_off: mute
then:
- if:
condition:
lambda: return id(wake_word_engine_location).state == "In Home Assistant";
then:
- lambda: id(voice_asst).set_use_wake_word(true);
- voice_assistant.start_continuous:
- if:
condition:
lambda: return id(wake_word_engine_location).state == "On device";
then:
- micro_wake_word.start
- lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
else:
- lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id};
- lambda: id(init_in_progress) = false;
- script.execute: reset_led
on_client_disconnected:
- if:
condition:
lambda: return id(wake_word_engine_location).state == "In Home Assistant";
then:
- lambda: id(voice_asst).set_use_wake_word(false);
- voice_assistant.stop:
- if:
condition:
lambda: return id(wake_word_engine_location).state == "On device";
then:
- micro_wake_word.stop
- lambda: id(voice_assistant_phase) = ${voice_assist_not_ready_phase_id};
- script.execute: reset_led
script:
- id: reset_led
then:
- if:
condition:
lambda: return !id(init_in_progress);
then:
- if:
condition:
lambda: return id(voice_assistant_phase) == ${voice_assist_listening_phase_id};
then:
- light.turn_on:
id: led_ring
blue: 100%
red: 0%
green: 0%
brightness: 100%
effect: wakeword
- if:
condition:
lambda: return id(voice_assistant_phase) == ${voice_assist_thinking_phase_id};
then:
- light.turn_on:
id: led_ring
blue: 100%
red: 100%
green: 0%
brightness: 100%
effect: Working
- delay: 100ms
- if:
condition:
lambda: return id(voice_assistant_phase) == ${voice_assist_replying_phase_id};
then:
- light.turn_on:
id: led_ring
blue: 100%
red: 0%
green: 0%
brightness: 100%
effect: Working
- if:
condition:
lambda: return id(voice_assistant_phase) == ${voice_assist_idle_phase_id};
then:
- light.turn_on:
id: led_ring
blue: 100%
red: 0%
green: 0%
brightness: 40%
effect: none
- delay: 200ms
- if:
condition:
lambda: return id(voice_assistant_phase) == ${voice_assist_not_ready_phase_id};
then:
- light.turn_on:
id: led_ring
blue: 40%
red: 100%
green: 0%
effect: Slow Pulse
- if:
condition:
lambda: return id(voice_assistant_phase) == ${voice_assist_error_phase_id};
then:
- light.turn_on:
id: led_ring
blue: 0%
red: 100%
green: 0%
brightness: 100%
effect: none
- if:
condition:
lambda: return id(voice_assistant_phase) == ${voice_assist_muted_phase_id};
then:
- light.turn_off: led_ring
else:
- light.turn_on:
id: led_ring
blue: 0%
red: 100%
green: 0%
effect: Fast Pulse
light:
- platform: esp32_rmt_led_strip
id: led_ring
name: "Led strip light"
pin: GPIO33
num_leds: 12
rmt_channel: 0
rgb_order: GRB
chipset: ws2812
default_transition_length: 0s
effects:
- pulse:
name: "Pulse"
transition_length: 300ms
update_interval: 300ms
min_brightness: 50%
max_brightness: 100%
- addressable_twinkle:
name: "Working"
twinkle_probability: 10%
progress_interval: 5ms
- addressable_color_wipe:
name: "Wakeword"
colors:
- red: 28%
green: 100%
blue: 90%
num_leds: 12
add_led_interval: 40ms
reverse: false
- addressable_color_wipe:
name: "Connecting"
colors:
- red: 60%
green: 60%
blue: 60%
num_leds: 12
- red: 60%
green: 60%
blue: 0%
num_leds: 12
add_led_interval: 100ms
reverse: true
- addressable_color_wipe:
name: "Thinking"
colors:
- red: 1%
green: 90%
blue: 99%
num_leds: 2
- red: 13%
green: 17%
blue: 87%
num_leds: 6
add_led_interval: 75ms
reverse: true
- pulse:
name: "Slow Pulse"
transition_length: 0.5s
update_interval: 1s
min_brightness: 0%
max_brightness: 100%
- pulse:
name: "Fast Pulse"
transition_length: 50ms
update_interval: 100ms
min_brightness: 50%
max_brightness: 100%
switch:
- platform: template
name: Mute
id: mute
optimistic: true
restore_mode: RESTORE_DEFAULT_OFF
entity_category: config
on_turn_off:
- if:
condition:
lambda: return !id(init_in_progress);
then:
- lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
- if:
condition:
not:
- voice_assistant.is_running
then:
- if:
condition:
lambda: return id(wake_word_engine_location).state == "In Home Assistant";
then:
- lambda: id(voice_asst).set_use_wake_word(true);
- voice_assistant.start_continuous
- if:
condition:
lambda: return id(wake_word_engine_location).state == "On device";
then:
- micro_wake_word.start
- script.execute: reset_led
on_turn_on:
- if:
condition:
lambda: return !id(init_in_progress);
then:
- lambda: id(voice_asst).set_use_wake_word(false);
- voice_assistant.stop
- micro_wake_word.stop
- lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id};
- script.execute: reset_led
- platform: restart
name: "${name} Restart"
select:
- platform: template
entity_category: config
name: Wake word engine location
id: wake_word_engine_location
optimistic: true
restore_value: true
options:
- In Home Assistant
- On device
initial_option: On device
on_value:
- wait_until:
lambda: return id(voice_assistant_phase) == ${voice_assist_muted_phase_id} || id(voice_assistant_phase) == ${voice_assist_idle_phase_id};
- if:
condition:
lambda: return x == "In Home Assistant";
then:
- micro_wake_word.stop
- delay: 500ms
- if:
condition:
switch.is_off: mute
then:
- lambda: id(voice_asst).set_use_wake_word(true);
- voice_assistant.start_continuous:
- if:
condition:
lambda: return x == "On device";
then:
- lambda: id(voice_asst).set_use_wake_word(false);
- voice_assistant.stop
- delay: 500ms
- micro_wake_word.start
globals:
- id: init_in_progress
type: bool
restore_value: false
initial_value: "true"
- id: voice_assistant_phase
type: int
restore_value: false
initial_value: ${voice_assist_not_ready_phase_id}
- id: speaker_volume
type: int
restore_value: yes
initial_value: "70" # Initial volume level (0-100)
button:
- platform: template
name: "Volume Up"
id: btn_volume_up
on_press:
then:
- lambda: |-
if (id(speaker_volume) < 100) {
id(external_speaker).volume_up();
}
- platform: template
name: "Volume Down"
id: btn_volume_down
on_press:
then:
- lambda: |-
if (id(speaker_volume) > 0) {
id(external_speaker).volume_down();
}
binary_sensor:
- platform: template
name: "Volume Up"
id: btn_vol_up
publish_initial_state: True
on_press:
then:
- lambda: |-
if (id(speaker_volume) < 100) {
id(external_speaker).volume_up();
}
- platform: template
name: "Volume Down"
id: btn_vol_down
publish_initial_state: True
on_press:
then:
- lambda: |-
if (id(speaker_volume) > 0) {
id(external_speaker).volume_down();
}
- platform: template
name: "Set"
id: btn_set
publish_initial_state: True
- platform: template
name: "Play"
id: btn_play
publish_initial_state: True
- platform: template
name: "Mode"
id: btn_mode
publish_initial_state: True
- platform: template
name: "Record"
id: btn_record
publish_initial_state: True
on_press:
- voice_assistant.start:
- light.turn_on:
id: led_ring
blue: 0%
red: 0%
green: 100%
brightness: 100%
effect: "Wakeword"
# Status connection
- platform: status
name: "Status"
sensor:
- id: button_adc
platform: adc
internal: true
pin: 39
attenuation: 12db
update_interval: 15ms
filters:
- median:
window_size: 5
send_every: 5
send_first_at: 1
- delta: 0.1
on_value_range:
- below: 0.55
then:
- binary_sensor.template.publish:
id: btn_vol_up
state: ON
- above: 0.65
below: 0.92
then:
- binary_sensor.template.publish:
id: btn_vol_down
state: ON
- above: 1.02
below: 1.33
then:
- binary_sensor.template.publish:
id: btn_set
state: ON
- above: 1.43
below: 1.77
then:
- binary_sensor.template.publish:
id: btn_play
state: ON
- above: 1.87
below: 2.15
then:
- binary_sensor.template.publish:
id: btn_mode
state: ON
- above: 2.25
below: 2.56
then:
- binary_sensor.template.publish:
id: btn_record
state: ON
- above: 2.8
then:
- binary_sensor.template.publish:
id: btn_vol_up
state: OFF
- binary_sensor.template.publish:
id: btn_vol_down
state: OFF
- binary_sensor.template.publish:
id: btn_set
state: OFF
- binary_sensor.template.publish:
id: btn_play
state: OFF
- binary_sensor.template.publish:
id: btn_mode
state: OFF
- binary_sensor.template.publish:
id: btn_record
state: OFF
# # Wifi signal
# - platform: wifi_signal
# name: "WiFi Signal"
# update_interval: 60s
# Generic volume sensor was used so it can be used in the esp_adf_speaker to get volume into HA
- platform: template
id: generic_volume_sensor
internal: true
- platform: template
name: "Current Volume"
accuracy_decimals: 0
id: scaled_volume_sensor
lambda: |-
// Scale the volume from 0-100 to 0-10
return id(generic_volume_sensor).state / 10.0; |
|
That's super interesting -- there must be fairly substantial hardware revisions between the early boards like yours and the newer ones. Like, the PA GPIO has changed, which suggests the board itself was a new layout, not just component swaps or tweaks in later revisions. I wonder if that's part of why some people are getting it to work and some aren't. @dwitgen's libraries didn't help on my V5 board -- they don't even compile, as they're clearly targeted at the earlier revisions. Reviewing the code, there's nothing unusual being done to initialize things, so I am starting to think fundamentally there's an ADF compatibility issue with the V4/V5 boards, and it has nothing to do with ESPHome or any of these external components. |
|
@dotorg I've got a Korvo V5.0 Board as well and although I am able to have it show up in my home assistant (LEDs and Button presses working), with a @abmantis code, there is no Audio playback and no received audio recordings. I sure hope you are more successful than me! Invested here and hoping for insights ;) |
|
My V5 works fine with everything but sound output. It's a little weird because i2s is pretty standardized, so it's a bit of a head scratcher why it isn't just working. There aren't any easily accessible test points to get a scope on to see if the problem is the DAC or the amp stages. I just know there's no output from the speaker or headphone outputs, as tested on an oscilloscope. The one thing I haven't tested yet is doing an Arduino framework test. The ES8311 is supported in Arduino, so that test not working may suggest it's an issue getting the amp stage powered up, vs an ESP-IDF/ADF problem. It's been on my list to try, just haven't had time yet. Unfortunately the ADC chip used by the mic isn't supported there, so it isn't a "solution", even if it started making the speaker work. It doesn't help that Espressif's docs are terrible, and a lot of the sample code doesn't actually work. It's clear that whole line of boards is largely abandoned, or at least was never really invested in to produce quality support materials. I suspect they thought there was a market for 3rd party voice assistant hardware, and the massive losses at Amazon and Google in those markets gave them pause. The same thing seems to have happened with their Matter/Thread bridge hardware stack, too. |
|
So, even with a basic Arduino sketch, I can't get output from the ES8311 on the V5 board -- simple sketches that work fine on other boards with the same electronics. I also found as soon as I enable support for it via i2c, the microphone also stops working. The schematic is a mess -- it has the i2c address wrong, or on the wrong sheet, for example. There's some feedback from the DAC back to the ADC, which I think is made so if you're playing music on the unit, the ADC can filter it out and still detect speech. I actually think either my board is defective or the design is -- I didn't realize the default firmware is supposed to respond with tones until I went digging in the source for it, and I hadn't remembered hearing them. I reflashed the factory firmware and, as expected, voice recognition worked but I got no audio output. So, I may have been spinning my wheels if it turns out the board itself just isn't working. |
|
@dotorg The V5 you are referencing is a different ESP32 chip than the ESP32-korvo-v1.1. Here is the link to the esp32-korvo-v1.1: https://github.com/espressif/esp-skainet/blob/master/docs/en/hw-reference/esp32/user-guide-esp32-korvo-v1.1.md and here is the link to the esp32-s3-korvo-1: https://github.com/espressif/esp-skainet/blob/master/docs/en/hw-reference/esp32s3/user-guide-korvo-1.md. So, there are differences in the board pins the microphone boards are the same though. I am not in a position to maintain repos and I am not really knowledgeable enough either. I am going to put the 2 board configs I have and the modified esp-adf components that appear to work. I would suggest pulling them in the event something changes, and I have to drop them, again I have no plans to unless they break. Here is a working yaml with links to the repos for the ESP32-S3_Korvo-1 that should work with the V5 board this same config can be used with the ESP32-Korvo-v1.1 board by changing the components: ref: and the esp_adf: board:, there are comments at both these lines to show what needs changed. one thing I forgot was there is also a difference in the ADC and LED pins between the 2 boards. I have updated this with comments on the lines for ADC and LED pins. |
|
@dwitgen s Code sample works for me on my V5.0 board. I get audio output from the headphone jack. Microphones are picking up sound and sending it to my Homeassistant instance. I can even switch wake words in HA. Thank you so much for posting, saved me some sleep deprived nights @dwitgen! 👍 🤟 |
Yes the mic is always streaming to HA as this is used for wake word, if you use on device wake word the mic will always be on but the stream is local for wake word detection. I had tried to get the on device wake word to work previously with no luck. When I was looking at my configs I seen the testing of on device wake word so I removed that from what I posted earlier. I cannot say about routing audio to another speaker. I had made some attempts at playing media to these and was able to get an https stream to partially work, it was very buggy and would likely take more skill than I have. There is also the possibility of multicast audio on esp32 which could then possibly used as an intercom again that would be quite a task to complete. I was able to get micro wake word to work. It is a little buggy as it does not like to switch back and forth correctly. It does seem to work if you change it then reset the device by the switch. This was tried on both the esp32-s3-korvo-1 and the esp32-korvo-1.1. Also had to modify the esphome micro_wake_word component. Mainly point to use the esp_adf_microphone used with the device instead of the esphome microphone. Not sure why but that was the only way I could get it to work. Also had an issue with the loop taking longer than 30ms and causing alot of errors so , made some changes to that as well. I will leave that as part of the repo but again I cannot guarantee it will be maintained or remain working. Below is the yaml for the esp32-s3-korvo-1 to work with on device wake word with comments for what is needed for the esp32-korvo-1.1 board: |
|
Interesting -- so that does work on my unit, although not for long. Something seems to trip it up and it occasionally will try to restart the mic while the mic is in the process of stopping, it gets an error and never tries again. Not a big deal -- the normal ESP-ADF support works fine with the mic, anyway. Looking through the branch, @dwitgen, can I ask why you re-implemented the 8311 support? Some of the code seems to be mostly cut-n-pasted from the ADF framework implementation, but because it wasn't forked from it, I can't tell what you needed to change. Since long-term getting a full ADF component working with this is better (both for long-term maintenance and so stuff like the media player is supported), I'm still trying to figure out what you're doing differently that works where the stock 8311 support doesn't. (I think there's an issue in ADF, as the 8311 support works fine on all my other boards but I do not get output even from any of the Espressif samples but I do from your implementation...) |
@dotorg So, this was and is all just testing At the time I was doing testing with the 8311 and esp_adf but that was not for esp_adf. I was just going back and forth with my testing is why it was in there and is not used at all. esp_adf handles the 8311 and 7210 chips in the back end I believe. The normal esp_adf support, I am assuming you mean this github://pr#5230. This should work as this pr has the esp32s3korvo1 board in the init.py. The changes I made to the esp_adf where to add the esp32korvo1 (not s3) board to the init.py and also made changes to the speaker to allow for volume controls. I was never able to get volume to function with the pr#5230, that may have been due to me not doing something correct in the yaml but I did get volume to work making changes to the esp_adf speaker code. Also changed how the pa was used. Again, not saying it is correct, but I disabled the PA on setup and enable the pa at speaker start then stop it after it is done. This seemed to help with speaker popping noises. No changes were done to the microphone. But with that being said there is some differences in the yaml between the esp32-s3-korvo and the esp32-korov 1.1 so I am going to paste that yaml seeing how this subject is for the esp32-korvo-v1.1 and not the esp32-s3-korvo-1. |
|
So because this subject is for the esp32-korvo-v1.1 I should have posted the yaml for that and not the esp32-s3-korvo-1. So here is the yaml for the esp32-korvo-v1.1 and again is just testing and may not work as desired but should play back audio from the speaker and headphone jack. This yaml is set to user the esp32-wrover-kit. |
|
@dwitgen I get this error while compiling in esphome: Do you have an idea how to fix / workaround this? |
|
@matze19999 Ignore the below, the latest esphome is a major change and they are using pioarduino and the configuration is quite different. I am not sure how you need to rollback but that is what I would suggest you do as the configuration as it sits will not work with the new esphome version. The latest ESPHome has updated the recommended IDF to 5.1.5. Try changing the version in the yaml. I have not updated so I cannot say for sure it will work. There is an issue see here: esphome/wake-word-voice-assistants#42 there was a post in this to revert ESPHome back to 2024.11.3. to this |
|
Thank you, I got it working with your hint: |
and others
Describe the problem you have/What new integration you would like
Add support for the espressif esp32-korvo-v1.1, see documentation here: https://github.com/espressif/esp-skainet/blob/master/docs/en/hw-reference/esp32/user-guide-esp32-korvo-v1.1.md
Please describe your use case for this integration and alternatives you've tried:
This will provide another option for a voice assistant with all the needed features built-in.
Additional context
This board is available, relatively inexpensive, and has a microphone array, leds, and speaker output.
The text was updated successfully, but these errors were encountered: