Only seems to pick up the first thing I say after starting service? #38

buffeh · 2024-05-24T15:43:03Z

buffeh
May 24, 2024

Hi,

Maybe I'm missing something. But I start the GPTHome service. Then in the event logs I see it ticking along with "INFO:functions:Could not understand audio, waiting for a new phrase..." if I'm making random noises or speaking without using the wake word - which is completely normal and fine. And if I turn on debug I see the usual "DEBUG:openai:message='Request to OpenAI API' method=get path=https://api.openai.com/v1/models" type logs. All good.

If I say something to it using the wake word "i.e. computer what is a banana" then I can see that come up in the log ok as a success, i.e. "SUCCESS:functions:Heard: "what is a banana"" - I don't see anything about a response unless I turn on debug, then I see:

DEBUG:openai:message='Request to OpenAI API' method=post path=https://api.openai.com/v1/chat/completions
DEBUG:openai:api_version=None data='{"model": "gpt-3.5-turbo", "messages": [{"role": "system", "content": "You are a helpful assistant. "}, {"role": "user", "content": "Human: what does apricot\nAI:"}], "max_tokens": 100, "temperature": 0.7}' message='Post details'
DEBUG:urllib3.util.retry:Converted retries value: 2 -> Retry(total=2, connect=None, read=None, redirect=None, status=None)
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.openai.com:443
DEBUG:urllib3.connectionpool:https://api.openai.com:443 "POST /v1/chat/completions HTTP/1.1" 200 None

But I don't see the actual response to my query show up as a "Success" line - unless, I restart the GPTHome service, then the response seems to come through to the logs. If I don't restart the service then no further commands seem to be picked up (at least according to the logs). Once I restart the service, I can issue one command and then no more are accepted. I know it's made it over to OpenAI as I can see the usage against the API key go up.

Additionally I appear to have no audiable responses, but I don't know whether that is because something is going on with the audio with GPT-Home (speaker-test works fine, I am using a mini speaker that plugs into the 3.5 port much like the one pictured against the project), or whether I'm just not getting any response which is why I hear nothing.

Any help appreciated

EDIT: It's like it stops accepting commands as it never gets the answer back from OpenAI - the query reaches openAI, as I can see the API usage increment. Then no answer comes back until I stop and restart the service, the answer comes back when I do that (but I don't hear anything come out of my speaker) and I am able to issue another command.

Answered by buffeh

May 25, 2024

Bingo - got it working. One thing in asound.conf needed changing. The line "#pcm.!default { type plug; slave.pcm "dmix0"; }" needed to be commented out, and replaced with "pcm.!default { type hw; card 0; }" - once that was done everything started working immediately!

Thank you very much for your help with this, I would have had no idea where to look at all if it wasn't for you. Out of curiosity any idea why my asound.conf had what it did for the default pcm device?

View full answer

judahpaul16 · 2024-05-24T16:36:15Z

judahpaul16
May 24, 2024
Maintainer

When SUCCESS:functions:Heard: "what is a banana" logs, do you hear that message spoken audibly? Also are you using Wi-Fi or Ethernet? My guess is the loop could be hanging as it waits for the response from OpenAI. About how long after speaking your command do you have to wait to hear SUCCESS:functions:Heard: "your message"?

Edit: Also, does this behavior repeat with other models?

11 replies

judahpaul16 May 24, 2024
Maintainer

No prob. Let's try running the app directly and see if we run into any weird audio errors. This might be a little tricky because ALSA likes to vomit out a walls of logs but lets see. First run docker exec -it gpt-home bash to move into the docker container. Then stop the app and run it manually:

supervisorctl stop app
source /env/bin/activate # activate the virtual environment
cd src # move into src folder
python /app/src/app.py 2>/dev/null # run the app with stderr redirected to /dev/null

Try speaking the commands again. If nothing appears in the terminal, Ctrl+C and try running without 2>/dev/null:

python /app/src/app.py

I should note, there will be a wall of logs that may not necessarily be related to your issue. Generally you can ignore the warning about jack server not running because it isn't needed here.

buffeh May 24, 2024
Author

Hi, thank you

Nothing came up when I ran it with dev/null and spoke to it. Without dev/null theres a lot of this sort of thing in the logs:

ALSA lib conf.c:5180:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5703:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM spdif
ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm_dmix.c:972:(snd_pcm_dmix_open) The dmix plugin supports only playback stream
ALSA lib pcm_dmix.c:972:(snd_pcm_dmix_open) The dmix plugin supports only playback stream
ALSA lib pcm_dmix.c:999:(snd_pcm_dmix_open) unable to open slave
ALSA lib pcm_dmix.c:972:(snd_pcm_dmix_open) The dmix plugin supports only playback stream
ALSA lib pcm_dmix.c:999:(snd_pcm_dmix_open) unable to open slave
ALSA lib pcm_dmix.c:972:(snd_pcm_dmix_open) The dmix plugin supports only playback stream
Cannot connect to server socket err = No such file or directory

ALSA lib confmisc.c:1369:(snd_func_refer) Unable to find definition 'cards.0.pcm.front.0:CARD=0'
ALSA lib conf.c:5180:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5703:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM front
ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib confmisc.c:1369:(snd_func_refer) Unable to find definition 'cards.0.pcm.surround51.0:CARD=0'
ALSA lib conf.c:5180:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5703:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM surround21
ALSA lib confmisc.c:1369:(snd_func_refer) Unable to find definition 'cards.0.pcm.surround51.0:CARD=0'
ALSA lib conf.c:5180:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5703:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM surround21
ALSA lib confmisc.c:1369:(snd_func_refer) Unable to find definition 'cards.0.pcm.surround40.0:CARD=0'

ALSA lib conf.c:5180:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5703:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM spdif
ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm_dmix.c:972:(snd_pcm_dmix_open) The dmix plugin supports only playback stream
ALSA lib pcm_dmix.c:972:(snd_pcm_dmix_open) The dmix plugin supports only playback stream
ALSA lib pcm_dmix.c:999:(snd_pcm_dmix_open) unable to open slave
ALSA lib pcm_dmix.c:972:(snd_pcm_dmix_open) The dmix plugin supports only playback stream
ALSA lib pcm_dmix.c:999:(snd_pcm_dmix_open) unable to open slave
ALSA lib pcm_dmix.c:972:(snd_pcm_dmix_open) The dmix plugin supports only playback stream
Cannot connect to server socket err = No such file or directory

judahpaul16 May 24, 2024
Maintainer

You mentioned speaker test worked but are you able to play one of the preinstalled alsa sounds with aplay?

aplay /usr/share/sounds/alsa/Front_Center.wav

buffeh May 24, 2024
Author

Ahh interesting. If I run aplay /usr/share/sounds/alsa/Front_Center.wav while in the docker container, it says "Playing WAVE '/usr/share/sounds/alsa/Front_Center.wav' : Signed 16 bit Little Endian, Rate 48000 Hz, Mono" but no audio ever actually plays, and it just doesn't finish executing either, and I have to control-C to get back to the command line, and then it says "aplay: pcm_write:2127: write error: Interrupted system call"

If I run aplay /usr/share/sounds/alsa/Front_Center.wav while NOT in the docker container, it works fine, I can hear it, and it puts me back at command prompt once the wav has been played.

And I also just tried speaker-test - same thing - in the Docker container it doesn't play anything, and when I quit out of it, it says "^CWrite error: -4,Interrupted system call
xrun_recovery failed: -4,Interrupted system call
Transfer failed: Interrupted system call" - and speaker-test works fine outside of the docker container.

I am thinking that this is the issue, it's trying to play the audio when I speak the command, but it hangs forever instead of playing the audio. Fix the audio and possibly fix the whole issue?

judahpaul16 May 24, 2024
Maintainer

Yeah I think we've isolated the problem to ALSA. Let's try updating the user permissions inside the docker container:

sudo usermod -aG audio your_container_user

then try running aplay again within the container.

Also check that asound.conf exists in the container and has stuff in it:

cat /etc/asound.conf

buffeh · 2024-05-24T23:47:15Z

buffeh
May 24, 2024
Author

Hi,
inside the container it says I'm "root@GPTHome" so inside the container I ran "usermod -aG audio root" and outside the container I'm "buffeh@GPTHome" so outside the container I ran "sudo usermod -aG audio buffeh"

I rebooted and still can't play sound from inside the container.

asound.conf exists in the container:

pcm.!default { type plug; slave.pcm "dmix0"; }
ctl.!default { type hw; card 0; }
pcm.dmix0 { type dmix; ipc_key 1024; ipc_perm 0666; slave { pcm "hw:0,0"; channels 2; period_time 0; period_s>
pcm.!hdmi { type plug; slave.pcm "dmix1"; }
ctl.!hdmi { type hw; card 1; }
pcm.dmix1 { type dmix; ipc_key 1025; ipc_perm 0666; slave { pcm "hw:1,0"; channels 2; period_time 0; period_s>

Do I need to rebuild something after updating the user permissions?

4 replies

judahpaul16 May 24, 2024
Maintainer

What's the output of aplay -l within the container?

buffeh May 25, 2024
Author

Hi,
I think we might be thinking along similar lines!
That output is:
**** List of PLAYBACK Hardware Devices ****
card 0: Headphones [bcm2835 Headphones], device 0: bcm2835 Headphones [bcm2835 Headphones]
Subdevices: 8/8
Subdevice #0: subdevice #0
Subdevice #1: subdevice #1
Subdevice #2: subdevice #2
Subdevice #3: subdevice #3
Subdevice #4: subdevice #4
Subdevice #5: subdevice #5
Subdevice #6: subdevice #6
Subdevice #7: subdevice #7

So speaker-test within the container doesn't work when no parameters are set. But just now I found out that it DOES WORK within the container if device is specified as such:

speaker-test -Dplughw:0,0

buffeh May 25, 2024
Author

Bingo - got it working. One thing in asound.conf needed changing. The line "#pcm.!default { type plug; slave.pcm "dmix0"; }" needed to be commented out, and replaced with "pcm.!default { type hw; card 0; }" - once that was done everything started working immediately!

Thank you very much for your help with this, I would have had no idea where to look at all if it wasn't for you. Out of curiosity any idea why my asound.conf had what it did for the default pcm device?

Answer selected by judahpaul16

judahpaul16 May 25, 2024
Maintainer

Likely because I was messing around with asound.conf when I was testing on the Raspberry Pi Zero 2 W with the Raspiaudio HAT. See the setup script lines 124-132:

# Create ALSA config (asound.conf, adjust as needed)
sudo tee /etc/asound.conf > /dev/null <<EOF
pcm.!default { type plug; slave.pcm "dmix0"; }
ctl.!default { type hw; card 0; }
pcm.dmix0 { type dmix; ipc_key 1024; ipc_perm 0666; slave { pcm "hw:0,0"; channels 2; period_time 0; period_size 1024; buffer_size 4096; rate 48000; } bindings { 0 0; 1 1; } }
pcm.!hdmi { type plug; slave.pcm "dmix1"; }
ctl.!hdmi { type hw; card 1; }
pcm.dmix1 { type dmix; ipc_key 1025; ipc_perm 0666; slave { pcm "hw:1,0"; channels 2; period_time 0; period_size 1024; buffer_size 4096; rate 48000; } bindings { 0 0; 1 1; } }
EOF

I'll put in a patch to make a simpler config that should work for most cases.

# Create ALSA config (asound.conf, adjust as needed)
sudo tee /etc/asound.conf > /dev/null <<EOF
pcm.!default { type hw card 0 }
ctl.!default { type hw card 0 }
EOF

taherafridi · 2024-05-27T15:45:44Z

taherafridi
May 27, 2024

Hi, I'm not sure if it's the same issue but when I run:

aplay /usr/share/sounds/alsa/Front_Center.wav

I hear no sound, there a small pause then it comes back to the prompt. When I run it with sudo, I hear the wav file.

sudo aplay /usr/share/sounds/alsa/Front_Center.wav

Is this normal and does this affect the speaker output of the project? I can't hear any feedback from it yet.

3 replies

judahpaul16 May 27, 2024
Maintainer

Sounds like a permissions issue (no pun intended). First try checking the device volume with alsamixer just in case. Add your user to the audio group.

sudo usermod -a -G audio $USER

Test aplay again to see if this fixed it. If still no luck try rebooting. If still no luck check the permissions on ls -l /usr/share/sounds/alsa/Front_Center.wav and report back.

taherafridi May 28, 2024

Thanks, This is the output of the ls command:

cosmo@cosmo:~ $ ls -l /usr/share/sounds/alsa/Front_Center.wav
-rw-r--r-- 1 root root 137134 Apr 16  2021 /usr/share/sounds/alsa/Front_Center.wav

judahpaul16 May 28, 2024
Maintainer

Is your user in the audio group? You can check by running groups.

mowtschan · 2024-08-10T11:52:31Z

mowtschan
Aug 10, 2024

I have exactly the opposite of what @taherafridi mentioned.
If I try:
aplay /usr/share/sounds/alsa/Front_Center.wav
I hear sound but if I try
sudo aplay /usr/share/sounds/alsa/Front_Center.wav
I hear no sound.
And from docker container (docker exec -it gpt-home aplay /usr/share/sounds/alsa/Front_Center.wav) I also don't hear sound :-(

I'm trying to get it work on card 3 which is a usb audio speaker:

root@raspi:/app# cat /etc/asound.conf
pcm.!default { type hw card Headphones device 0 }
ctl.!default { type hw card Headphones }

root@raspi:/app# aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: vc4hdmi0 [vc4-hdmi-0], device 0: MAI PCM i2s-hifi-0 [MAI PCM i2s-hifi-0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: vc4hdmi1 [vc4-hdmi-1], device 0: MAI PCM i2s-hifi-0 [MAI PCM i2s-hifi-0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 2: Headphones [bcm2835 Headphones], device 0: bcm2835 Headphones [bcm2835 Headphones]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7
card 3: UACDemoV10 [UACDemoV1.0], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

0 replies

judahpaul16 · 2024-08-10T14:03:17Z

judahpaul16
Aug 10, 2024
Maintainer

Does aplay -D plughw:3,0 /usr/share/sounds/alsa/Front_Center.wav produce a sound within the docker container? If so you may want to alter /etc/asound.conf to something like this:

pcm.!default { type hw card 3 device 0 }
ctl.!default { type hw card 3 }

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Only seems to pick up the first thing I say after starting service? #38

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 5 comments 18 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Only seems to pick up the first thing I say after starting service? #38

buffeh May 24, 2024

Replies: 5 comments · 18 replies

judahpaul16 May 24, 2024 Maintainer

judahpaul16 May 24, 2024 Maintainer

buffeh May 24, 2024 Author

judahpaul16 May 24, 2024 Maintainer

buffeh May 24, 2024 Author

judahpaul16 May 24, 2024 Maintainer

buffeh May 24, 2024 Author

judahpaul16 May 24, 2024 Maintainer

buffeh May 25, 2024 Author

buffeh May 25, 2024 Author

judahpaul16 May 25, 2024 Maintainer

taherafridi May 27, 2024

judahpaul16 May 27, 2024 Maintainer

taherafridi May 28, 2024

judahpaul16 May 28, 2024 Maintainer

mowtschan Aug 10, 2024

judahpaul16 Aug 10, 2024 Maintainer

buffeh
May 24, 2024

Replies: 5 comments 18 replies

judahpaul16
May 24, 2024
Maintainer

judahpaul16 May 24, 2024
Maintainer

buffeh May 24, 2024
Author

judahpaul16 May 24, 2024
Maintainer

buffeh May 24, 2024
Author

judahpaul16 May 24, 2024
Maintainer

buffeh
May 24, 2024
Author

judahpaul16 May 24, 2024
Maintainer

buffeh May 25, 2024
Author

buffeh May 25, 2024
Author

judahpaul16 May 25, 2024
Maintainer

taherafridi
May 27, 2024

judahpaul16 May 27, 2024
Maintainer

judahpaul16 May 28, 2024
Maintainer

mowtschan
Aug 10, 2024

judahpaul16
Aug 10, 2024
Maintainer