Rebooting every few seconds?

[hpmueller1971]

Hi,

i successfully installed on an Atom Echo, it seems to work (ping, webui reachable), but it keeps rebooting every few seconds in a loop when in local hotword mode.

`ets Jun 8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 188777542, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:1044
load:0x40078000,len:8896
load:0x40080400,len:5828
entry 0x400806ac
BootiM5Atom initializing...Loading configuration
{
"mqtt_host": "10.123.1.2",
"mqtt_port": 1883,
"mqtt_user": "",
"mqtt_pass": "",
"mqtt_valid": true,
"mute_input": false,
"mute_output": false,
"amp_output": 0,
"brightness": 30,
"hotword_brightness": 100,
"hotword_detection": 0,
"volume": 100,
"gain": 5
}
Enter WifiDisconnected
Total heap: 270760
Free heap: 196232
Enter WifiConnected
Connected to Wifi with IP: 10.123.1.33
Enter MQTTDisconnected
Enter MQTTConnected
Connected as satellite
Enter Idle

Backtrace: 0x4008ca58:0x3ffbe170 0x4008cc89:0x3ffbe190 0x400ec4b3:0x3ffbe1b0 0x40084ab9:0x3ffbe1d0 0x4000bfed:0x3ffeb550 0x4008a181:0x3ffeb560 0x400896e5:0x3ffeb580 0x400d2e75:0x3ffeb5a0 0x40089015:0x3ffeb820

Rebooting...
ets Jun 8 2016 00:22:57
`

The Backtrace looks the same every time. Should "local hotword" even work yet...?

Any ideas?

regards,

/hp

[sp33dsk8]

I'm not working on the project but it appears since Satellite.cpp v7.0 hotword has been disabled locally

[Romkabouter]

Local hotword detection does not work on the latest version due to the fact that the newest esp library does not like the WakeNet library used.

This should not cause the reboot however, so I will look into this.

[hpmueller1971]

Ah, thanks guys, i missed that in the commit-log! So i play with the remote-wakeword detection for the time being... Does the fact, that you didnt remove the web-option mean, that there is hope for the WakeNet integration (this is also from espressif, right?), or is WakeNet dead for the ESP32?

[Romkabouter]

WakeNet development seems to have stopped. I was working in release 6, but the network stability was less good and also the M5 is not supported with that version (actually only the Matrix Voice is and that is the whole reason for the rewrite)

I am hoping on either a new WakeNet library or a Porcupine library. Untill then.. no local hotword ;(
Maybe it is best to remove the setting for now indeed. App still should not crash though.

[flatsiedatsie]

I'm out of my depth, but I think the issue might be related to MQTTDisconnected being called both by the state machine and by the run section:

  void run(void) override {
    //Serial.print("in run");
    if (audioServer.connected() && asyncClient.connected()) {
      transit<MQTTConnected>();
    } else {
      currentMillis = millis();
      if (currentMillis - startMillis > 5000) {
        Serial.println("Connect failed, retry");
        transit<MQTTDisconnected>();
      }      
    }
  }

and

  void react(MQTTDisconnectedEvent const &) override { 
    transit<MQTTDisconnected>();
  }

So there are two places that try to react to a disconnection by starting the reconnect process?

[Romkabouter]

No, it only reacts to events when in a state.

So what happens in the case you mention, the MQTTDisconnected state transits to itself when after 5 seconds no connection is succesfull.
There is no MQTTDisconnectedEvent fired.

That event is ONLY fired in the I2S task, whenever the code detects that the audioserver is disconnected.

[flatsiedatsie]

Thanks. I'm trying to get it to work, but I'm a bit out of my depth. So far I've gotten it to stay in Idle without crashing. I'm seeing some things that I don't understand.

Still seeing this double MQTTDisconnected after the I2STask notices an MQTT disconnect, followed by a crash.

onnected to Wifi with IP: 192.168.2.137
WifiConnected entry complete, moving to MQTTDisconnected
Enter MQTTDisconnected
MQTT was not initialized yet.
past setClientId
192.168.2.167
1883
clientID = atomechoAudio
past asyncclient.connect
past audioServer connect, reached end of mqttdisconnected init
end of setupEnter MQTTConnected
Connected as atomecho
subscribed to topics
Enter Idle
Still Idle
start streaming
past stream
end of idle
-I2S believes audioserver is disconnected -> sending MQTTDisconnectedEvent()
idle react to MQTTDisconnectedEvent
Enter MQTTDisconnected
Enter MQTTDisconnected
- had to disconnect async client
past setClientId
past setClientId
192192.168.2.167
1883
.168.2.167
1883
clientID = atomechoAudio
clientID = atomechoAudio
past asyncclient.connect
past asyncclient.connect

Backtrace: 0x4008c728:0x3ffbe170 0x4008c959:0x3ffbe190 0x400e7803:0x3ffbe1b0 0x40084789:0x3ffbe1d0 0x400819cf:0x3ffddef0 0x400826bf:0x3ffddf10 0x4008126f:0x3ffddf30 0x400dcf4d:0x3ffddf50 0x400dcfc6:0x3ffddf90 0x400d1e46:0x3ffddfc0 0x400d1e87:0x3ffde050 0x400d1f78:0x3ffde070 0x400d27e9:0x3ffde090 0x40088ce5:0x3ffde510

If I do a test push of a message in the I2sTask, it arrives if I send a fake string. But the actual audio data never arrives.

            audioServer.publish(audioFrameTopic.c_str(),(uint8_t *)payload, sizeof(payload)); // <- this does not arrive
            //audioServer.publish(audioFrameTopic.c_str(),"x", 1); // <- this arrives 

Even when sending these fake packets, I2sTask still notices that the audioServer is disconnected. I'm not 100% sure it actually is.

Not receiving audio data might be what Snips Watch is complaining about here, but I'm not sure. I believe this is after pressing the button:

[13:18:31] [Dialogue] was asked to start a session on site atomecho
[13:18:31] [Asr] was asked to stop listening on site atomecho
[13:18:31] [Hotword] was asked to toggle itself 'off' on site atomecho
[13:18:31] [Dialogue] session with id '110943cd-8fea-40ac-9a15-2f990279144f' was started on site atomecho
[13:18:31] [Asr] was asked to listen on site atomecho
[13:18:47] [Dialogue] session with id '110943cd-8fea-40ac-9a15-2f990279144f' was ended on site atomecho. The session was ended because one of the component didn't respond in a timely manner
[13:18:47] [Asr] was asked to stop listening on site atomecho
[13:18:47] [Hotword] was asked to toggle itself 'on' on site atomecho

I suspect Rhasspy has added the reason attribute to the spec. I tried removing that check to make it Snips compatible again:

        if (root["siteId"] == SITEID){
          send_event(IdleEvent());
          if(root.containsKey("reason")
            && root["reason"] == "dialogueSession") {
              //send_event(IdleEvent());
          }
        }

[flatsiedatsie]

I checked, and the microphone data does exist. It just doesn't see to be sent over.

82737070442008765866910210911632160001010128620001250020160100971169702007923681236762368023688236822368323682236882368623684236782369323690236103236992369223610423683236802369123684236882368223695236842361042361042369623693236952369723682236902369323683236102236972369223610323694236104236104236912368623696236102236962361032361092369723610523610923696236106236982361172361132361092361082361112361122361022361012361052361002361012369323610323610023612123611523612123612223612923612123614923613523613523614023613523614023611923612223612323611123612323612523612323612523612123613023611423611723611923611323612523612923612823612823612623614023613223613923614023613123612223613323613323612923610223611523687236812369423690236952369223684236852369523694236922369423672236812367023666236722368023682236762367423679236752366823691236792366823690236682367923658236712366523657236612366323638236482364223636236412364723626236352362323623236282362123630236242363023633236392363123634236412362823631236262363723662361723632366236255235023610236253235323625523511236423692361923652368236122361623613236523672365236523642362522357236623622362502352236122362532358236823632361023612367236222369236342362023642236332362923627236242362023617236172362462358236223672361823611236212363823620236192367236152361423610236232361823614236132361123616236523617236232361823616236122363123626236252362623621236232362532355236
8273707044200876586691021091163216000101012862000125002016010097116970200217236218236229236211236220236237236214236216236216236224236207236219236223236214236225236220236232236219236223236221236212236210236215236207236212236215236229236225236240236226236222236231236217236218236221236216236236236221236237236242236238236240236215236237236205236211236220236207236217236223236214236217236202236217236212236196236197236207236208236199236210236210236207236203236230236224236221236221236211236221236210236208236202236209236204236205236191236189236205236196236192236194236193236193236187236193236190236194236170236175236184236168236186236187236190236187236155236152236151236156236145236154236157236151236166236160236166236174236163236168236171236165236180236182236182236185236187236185236191236181236197236208236200236192236204236204236194236184236199236197236199236197236198236199236220236212236207236221236225236222236219236222236217236211236211236212236220236212236218236210236224236218236223236222236223236232236227236237236222236223236232236229236220236228236233236221236218236222236211236214236214236207236219236219236210236198236197236203236182236202236177236187236197236188236206236203236214236213236216236211236203236212236211236216236201236221236212236214236215236217236210236205236211236212236216236212236211236218236228236219236225236229236226236225236223236220236210236213236212236213236212236218236205236215236204236206236204236201236206236196236205236216236222236210236222236221236210236209236227236210236233236221236219236225236222236224236214236222236232236220236208236222236219236219236211236220236
8273707044200876586691021091163216000101012862000125002016010097116970200226236198236228236222236218236227236194236214236192236191236181236193236177236178236174236180236175236176236180236171236203236176236196236198236211236201236215236209236212236216236200236212236190236200236186236186236199236192236190236181236199236197236205236205236208236201236196236205236187236196236190236197236204236206236204236206236202236213236188236191236219236204236202236217236202236210236204236210236214236214236210236216236210236219236217236212236227236212236222236222236206236202236196236193236210236202236203236197236212236214236218236212236191236211236197236192236207236204236210236211236214236220236205236203236203236188236204236200236192236186236188236186236193236195236196236190236212236205236200236201236226236198236228236222236218236227236194236214236192236191236181236193236177236178236174236180236175236176236180236171236203236176236196236198236211236201236215236209236212236216236200236212236190236200236186236186236199236192236190236181236199236197236205236205236208236201236196236205236187236196236190236197236204236206236204236206236202236213236188236191236219236204236202236217236202236210236204236210236214236214236210236216236210236219236217236212236227236212236222236222236223236219236231236232236232236240236224236226236215236218236209236219236196236212236215236214236209236217236204236213236227236211236227236220236218236217236206236208236201236200236206236213236213236203236216236221236214236211236205236212236193236201236200236201236198236204236212236204236185236211236184236187236185236180236195236186236
8273707044200876586691021091163216000101012862000125002016010097116970200139236126236143236134236148236149236143236150236133236145236150236141236164236154236183236169236174236174236171236158236167236168236169236177236166236180236167236170236186236179236183236182236178236182236177236178236188236182236168236174236174236170236172236174236172236176236161236175236155236158236140236148236145236150236143236136236140236152236123236130236144236151236154236141236147236138236143236140236137236139236135236150236130236135236127236127236134236119236126236127236136236133236147236142236136236156236137236135236116236129236124236108236126236129236129236135236129236125236127236133236142236134236129236137236142236140236138236138236133236131236132236125236126236131236139236139236131236140236146236149236139236126236143236134236148236149236143236150236133236145236150236141236164236154236183236169236174236174236171236158236167236168236169236177236166236180236167236170236186236179236183236182236178236182236177236178236188236182236168236174236174236170236172236174236172236176236161236175236155236158236140236148236145236150236143236136236140236152236123236130236144236151236154236141236145236139236157236139236150236152236158236163236152236159236146236140236140236142236155236146236156236169236147236146236164236158236149236160236145236146236155236149236150236159236137236141236159236154236165236170236158236169236152236150236156236148236144236145236151236150236160236162236155236161236140236146236133236141236130236141236130236129236135236133236136236140236151236136236153236151236138236141236131236138236145236141236
82737070442008765866910210911632160001010128620001250020160100971169702001302361282361252361272361102361102361182361152361332361292361332361332361312361302361232361402361272361222361282361352361382361322361342361322361432361342361462361462361322361272361322361322361302361412361182361272361362361272361272361162361142361162369323611023698236892369423697236112236117236105236102236992369423610323610323612723611823611123611023610023611123611223610423612023611523611623611723612023611023612223612523611523612423613823613023614123613423613723613723614423613523614123614623615023614423614623614323614123614023613323613223613223612423612823612723613223612723613323613223611823613023611523611723612123612923611623611823610823611423611623611523611723611623612923612623611423611823697236106236942369223610623692236117236121236122236121236115236119236103236108236103236101236105236992361052369923610523610823610823610223611223611223610723611623611323611823610623611223611223611823610823612823612623610123611823611323612323611923610823612123610223610123610323697236117236105236117236122236119236109236101236113236992369823610323610223694236107236105236108236125236114236132236117236132236115236128236116236117236121236120236124236119236129236132236129236127236134236115236128236117236114236123236117236127236116236142236135236134236135236129236132236128236138236137236134236133236134236130236122236114236114236118236126236131236117236128236121236107236110236114236104236116236123236120236115236140236136236130236128236125236127236110236110236118236115236133236129236133236133236131236130236123236140236
8273707044200876586691021091163216000101012862000125002016010097116970200432363723633236372363323623236232363223620236242364623631236522365223657236452365823661236702367023664236752365523661236732367023660236622366023654236452364723664236492365023654236452364223641236472363923633236362364323629236392362623627236462364223642236502365823659236532366123656236612364223651236392363823650236552365023648236432365023651236412364923639236472365823662236522365923672236532365123652236522364323647236602365723640236512363323647236482363723648236522365523644236542365223649236482365123650236472364523656236492365923651236562365423657236492366223655236672365623644236482363723645236432363723633236372363323623236232363223620236242364623631236522365223657236452365823661236702367023664236752365523661236732367023660236622366023654236452364723651236392365123652236462364923633236392365023641236552365123645236532364623640236502364523632236462363423645236312363923627236232363623626236192362523622236212361823613236262361923618236222363723614236432363723646236452363023650236222362923627236202363723628236262362423612236242361323614236322362923634236462363623636236182363723616236182362723633236302363623623236362363023629236292361623623236272369236182362523619236202362723628236202363423633236312363123634236362368236262362223616236202362323636236322363123629236
827370704420087658669102109116321600010101286200012500201601009711697020022223524123520423521023521923522823521623521623523123523523523123523523522223522223520923521523542363236523602362522355236223625523525523524923514236723616236122367236212361423617236112361723662366236249235248235102362472352522358236244235244235245235246235249235240235323625423513236223617236102361623620236112361123617236923611236112361623619236623625423511236251235242235249235244235250235236235123623623524123523823524823524023523823524223523023524423523623525123524323532362432350236252235252235254235236235244235219235229235203235200235216235204235220235217235228235224235221235226235227235222235239235223235238235238235222235227235239235231235239235239235237235253235226235224235230235234235242235230235254235254235241235248235248235247235217235230235222235216235222235207235233235223235240235241235249235249235112361236323672362236323624923502362452352472352492352236246235249235247235240235223235232235237235239235246235239235255235239235023625223592363236248235823623623524023524023524323524223523323524923502364236251235252235250235247235247235255235123623523524623532362502354236523692368236123625523525523512362352352432350236245235255235247235245235253235247235253235223625423525023512236623632360236023642367236240235252235241235239235251235242235254235223624823524223525023525123525223524923525323525223524723524123524023523223523023522523523323522923523423523623525123524123502362482352532354236
8273707044200876586691021091163216000101012862000125002016010097116970200180235168235182235183235171235176235156235165235156235150235160235154235164235165235156235163235164235163235175235171235164235168235175235162235181235179235167235180235142235155235149235150235152235153235150235149235157235150235166235158235165235170235184235166235200235186235198235189235180235196235159235174235184235182235178235185235186235190235188235193235213235194235226235224235233235232235221235229235226235219235214235214235213235209235212235211235206235210235203235205235193235203235193235195235199235198235202235202235191235199235178235187235183235189235172235184235161235174235160235156235170235171235151235148235163235157235168235163235174235170235188235185235173235183235184235175235162235191235164235169235155235155235161235147235168235160235175235177235173235165235180235183235170235176235174235170235186235176235179235179235184235179235157235162235165235163235177235176235173235163235170235182235141235160235149235149235143235146235150235146235142235147235143235144235153235150235151235154235151235149235149235145235150235148235153235150235146235150235141235144235138235135235148235147235157235145235151235155235146235142235151235157235148235155235167235160235166235157235172235162235171235171235175235175235170235176235169235173235189235175235191235196235183235190235198235185235186235196235179235178235191235185235185235187235174235175235186235183235182235177235195235186235206235204235199235212235195235198235189235195235178235188235184235182235186235174235187235193235164235175235172235176235177235182235176235174235
8273707044200876586691021091163216000101012862000125002016010097116970200772358223581235802351002359323511023598235131235114235129235120235137235132235125235134235128235129235121235133235119235121235113235127235119235114235122235120235121235120235109235117235126235117235124235130235128235131235127235128235136235129235153235146235131235132235129235127235117235122235116235116235117235110235129235127235127235139235109235124235130235116235137235129235132235136235115235117235952351032351022351002359223510623510623510323597235982358723598235102235822358723593235982358823586235932358223581235922358823594235902358723590235822358623580235792358423577235872358723510223510723511723511623511323512523595235111235952351002359323597235932359423592235942359723593235962351042351032351052351042351032351042351152359323597235102235992359423510223594235952359023587235932358123592235932358923591235912358623583235802358623583235692358423578235702357423576235842358323593235882351022359023510023510623584235832358123575235932358823597235101235109235104235942359723569235732358923571235692356023576235572358323568235872357123584235782358023576235882358423596235852358323597235842357623581235882357623577235682356823551235642354823555235512355623557235582356123560235692357023562235552357323567235892357523592235922357923587235792357923586235862358023580235822358623575235642351002358523591235992351022351032359223598235113235113235111235107235119235109235123235108235
8273707044200876586691021091163216000101012862000125002016010097116970200302353823531235222352223520235222352523524235182353423524235422353823540235452354323551235342354023534235212354523526235232352823539235362352623527235252352123534235362352623529235382353323551235522354423555235512354323533235522354223549235432354023553235482356823561235602356723550235452355723542235382354523540235382354123545235392354523545235322353823541235402354123541235402352723532235352352923538235442352923547235392354023531235292356323577235642356123554235612354523557235432353623548235512354323540235542355823551235572354223549235432354723558235472355023561235532355323563235622357223566235452356023552235532354723546235412354023543235472354723558235452354723552235362354323547235482354423547235482356523559235492355423551235502353723550235412354423533235452353623532235232352323535235222353423534235342353523529235222352823534235372353223535235392352323529235172352423512235202351523532351423572352923522235382352923521235392353323528235432353623537235352353223530235332354123535235432354423541235572354323556235482356823555235582355523576235632356823575235642357623554235582356023555235582356323570235662355923555235592357623557235532357523557235782357623573235632356423577235482355823553235612355123557235692355623563235652356223560235562355923563235622355523558235
827370704420087658669102109116321600010101286200012500201601009711697020022823422023421823422823423123421323422223422823420623421523421023420723421223420923421923421123424023422423422923423623422723423523423323423123425023423923425023424723410235255234023502359235223525023462352492342392342392342462342132342212342242342212342342342252342252342292342362342362342292342272342382342352342462342412342422342392342382342392342412342462342522342482342482342542342522342492342552342235132358235323518235323513235523525523413235923510235112351323518235312352323536235372353323539235302353423530235382352323527235132351423527235142352723524235212353423529235322351823525235623572351623516235122351623532357235523562352532346235255234255234223542359235254234723582352235223525423462352472342542348235623532353235323522353235623521235142352123522235122352023532357235172351323532358235292351023520235202352823522235322352623521235282351523562351823512235162351823524235222351623520235323552352542342552341235250234240234323524023423223425423425123424923424523424823424923425323425523423323423823423723423823423823424423424223424323422923422723423223422923422523422723423523423323482352552346235823513235523520235823515235142351323517235251234250234253234251234243234245234246234249234254234023524523442352235255234523512351623512235192353323515235222352552346235223542359235122352323522235222351823532235222353223526235
8273707044200876586691021091163216000101012862000125002016010097116970200208234198234206234205234187234193234201234197234215234219234210234209234216234214234211234214234204234213234209234211234203234199234190234189234189234195234194234185234200234206234197234205234212234201234201234214234211234212234206234209234206234206234201234196234203234197234205234206234195234201234183234188234195234196234190234186234232234222234230234227234218234221234227234220234229234240234221234214234226234228234222234228234216234223234209234221234208234215234206234215234215234201234218234211234205234208234198234197234194234198234203234198234203234198234211234204234216234210234218234218234209234219234208234208234181234197234180234187234189234190234191234191234196234196234182234187234201234194234200234196234200234188234216234202234216234207234199234214234191234186234211234209234217234211234215234221234210234209234209234201234208234214234195234209234190234182234186234193234200234188234199234206234208234197234208234204234211234217234199234214234197234198234207234207234206234207234208234208234205234203234219234218234213234214234227234215234208234210234206234203234212234203234200234215234198234189234192234202234191234182234184234189234191234186234178234188234190234178234172234184234194234181234176234198234173234178234168234171234161234155234177234156234172234170234164234182234168234164234172234180234173234169234157234164234160234165234164234167234173234155234190234171234194234188234194234197234188234192234174234188234179234177234198234196234190234201234183234193234167234180234150234155234166234162234166234183234

[Romkabouter]

The double MQTTDisconnected is indeed a bit strange. DId you put in the "idle react to MQTTDisconnectedEvent" in the MQTTConnected react as well? If not, please do and see if the other react is from the MQTTConnnected state
About the payloads not arriving, can you verify that in your load_settings.py the MQTT_MAX_PACKET_SIZE is set to 2000?

[flatsiedatsie]

DId you put in the "idle react to MQTTDisconnectedEvent" in the MQTTConnected react as well?

No I hadn't. I have now, and will keep an eye out.

can you verify that in your load_settings.py the MQTT_MAX_PACKET_SIZE is set to 2000?

I upload via the Arduino IDE. I do have #define MQTT_MAX_PACKET_SIZE 2000 in the code here.

[flatsiedatsie]

Found this in the PubSubClient issue queue:

knolleary/pubsubclient#810

[flatsiedatsie]

I've gotten a bit further!

can you verify that in your load_settings.py the MQTT_MAX_PACKET_SIZE is set to 2000

That was the issue. I added audioServer.setBufferSize(MQTT_MAX_PACKET_SIZE); in MQTTDisconnected entry, and now I receive actual audio data. Earlier on I only had #define MQTT_MAX_PACKET_SIZE 2000, but that wasn't enough in the Arduino IDE.

[Romkabouter]

Still reboots or is that issue also fixed?

[flatsiedatsie]

Yes. Altought I've made a number of changes (disabled color control and OTA), I think that's what fixed it. Also added:

    if(!I2StaskCreated){
      device->muteOutput(true);
      xEventGroupClearBits(audioGroup, STREAM);
      xEventGroupClearBits(audioGroup, PLAY);
      xTaskCreatePinnedToCore(I2Stask, "I2Stask", 30000, NULL, 3, NULL, 0);
      I2StaskCreated = true;
    }
    else{
      Serial.println("NOT CREATING I2S TASK AGAIN");
    }

because I noticed that task was created multiple times, and I wasn't sure if that was an issue.

Can't speak for @hpmueller1971 though.

At the risk of further hijacking this thread: my issue now is that I can't get Snips to react to "Hey Snips". I looked at the older code, but can't find which MQTT commands are needed to work with Snips.

I've previously created satellites based on the Pi Zero, and there pointing the satellite audio server to the main server 'automatically' made it work.

[koenvervloesem]

@flatsiedatsie For your Snips compatibility issues: have a look at https://rhasspy-hermes.readthedocs.io/en/latest/api.html. The documentation is not complete yet, but this should become the documentation of Rhasspy's dialect of the Snips Hermes protocol. It already should indicate which parts of the current protocol are Rhasspy-specific.

[flatsiedatsie]

The reboot issue re-appears if I disable sending audio data all the time. So if HW_REMOTE is not active, it crashes.

    if (config.hotword_detection == HW_REMOTE)
    {
      //xEventGroupSetBits(audioGroup, STREAM); // <- commenting this line causes crash
      Serial.println("idle started streaming");
    }

[flatsiedatsie]

I appreciate the suggestion Koen, but any concrete hints on what changed when Snips compatibility was removed would be greatly appreciated. I went over the code, but couldn't find anything concrete. Looking at the API won't help me either.

I don't use audio playing ability. I already can't get it to simply detect the hotword on the main server.

[koenvervloesem]

Sorry, I haven't touched Snips for a long time, so I can't give some concrete hints. Maybe ask on the Rhasspy forum? A couple of users there are still using Snips or have been using it very recently.

[flatsiedatsie]

I suspect it has something to do with bind. Snips has to be told atomecho@mqtt somehow.

Did the old code only work if you made changes to snips.toml on the main server?

[Romkabouter]

because I noticed that task was created multiple times, and I wasn't sure if that was an issue.

Pretty strange behaviour, are you using an ino file or my codebranch?
The I2Stask is created in the WifiDisconnected state, which I can see is an issue now when the state is re-entered. Good catch.

I was already investigating the HW_REMOTE and am making some changes.

I appreciate the suggestion Koen, but any concrete hints on what changed when Snips compatibility was removed would be greatly appreciated. I went over the code, but couldn't find anything concrete. Looking at the API won't help me either.

I don't use audio playing ability. I already can't get it to simply detect the hotword on the main server.

I think you need to check the void onMqttMessage function.
There are checks in the topics toggleOn and toggleOff, but the hotword is only triggers when a reason is "dialogueSession"
This was not the case with snips

Try changing this (on two places):

        if (root["siteId"] == SITEID 
          && root.containsKey("reason")
          && root["reason"] == "dialogueSession") {
            send_event(HotwordDetectedEvent());
        }

to this:

        if (root["siteId"] == SITEID) {
            send_event(HotwordDetectedEvent());
        }

The old code was:

        if (topicstr.find("toggleOff") != std::string::npos) {
            std::string payloadstr(payload);
            // Check if this is for us
            if (payloadstr.find(SITEID) != std::string::npos) {
                hotword_detected = true;
                xEventGroupSetBits(everloopGroup, EVERLOOP);  // Set the bit so the everloop gets updated
            }
        } else if (topicstr.find("toggleOn") != std::string::npos) {
            // Check if this is for us
            std::string payloadstr(payload);
            if (payloadstr.find(SITEID) != std::string::npos) {
                hotword_detected = false;
                xEventGroupSetBits(everloopGroup, EVERLOOP);  // Set the bit so the everloop gets updated
            }
        } 

[flatsiedatsie]

Try changing this (on two places)

I already had :-) See the bottom of this post a few posts up.

indeed, I couldn't find anything other than that. Still, Snips just isn't responding to "hey snips". In your experience, does the Atom Echo require saying it near or far from the device? You you have to speak loudly?

I was wondering if I had to use "atomecho@mqtt" somewhere, but couldn't find any mention in the code anywhere. As far as I can tell, the current code is similar to the older versions.

I also tried playing with snips.toml, but no luck.

[Romkabouter]

Ah sorry, missed that.
You need to disable the feedback sound, otherwise snips will send it and will wait for a finished message which is send after the playing of the feedback sound.
That is why you get the timeouts, check the original:
if (topicstr.find("playBytes") !=
part.

This is the finished message:
finishedMsg = "{\"id\":\"" + topicparts[4] + "\",\"siteId\":\"" + SITEID + "\",\"sessionId\":null}";
The id is that last part of the random topicid hermes/audioServer/siteID/playBytes/
That message should be send to hermes/audioServer/siteID/playFinished.

Also, in later version you need top implement "playBytesStreaming".

I suggest you disable the feedback to try this. I will try if I can spinup snips somewhere to help you out :)

[Romkabouter]

Hi,

i successfully installed on an Atom Echo, it seems to work (ping, webui reachable), but it keeps rebooting every few seconds in a loop when in local hotword mode.

/hp
Hi @hpmueller1971, I can reproduce this issue so I will fix it asap :)

[flatsiedatsie]

@Romkabouter thanks, that might be it. Don't worry about installing Snips, since Voco already drops a lot of Snips-isms too (satellites don't play audio streams). If you do really want you help out, you could try installing Voco on Webthings on a Raspberry Pi.

[Romkabouter]

I will do, intresting project :)

[flatsiedatsie]

@Romkabouter Alright, curious to hear your thoughts :-)

[Romkabouter]

@hpmueller1971 please check version 7.3:

https://github.com/Romkabouter/ESP32-Rhasspy-Satellite/releases/tag/v7.3

[Romkabouter]

@flatsiedatsie can you check 7.3 if you still see double creations of the I2Stask?
Should be fixed

[flatsiedatsie]

I don't use PlatformIO, so I can't test that for you, sorry.

I checked my code (which is essentially 7.1 in Arduino form), and it does already have the playBytes response code.

https://github.com/createcandle/voco-mini-satellite/blob/a2d4d5084bbed6f9412f4fb6c37a2b082287f5c6/StateMachine.hpp#L398

[Romkabouter]

Ah ok, the fixes are small so you should be able to put it in your sketch as well :)

[Romkabouter]

I checked my code (which is essentially 7.1 in Arduino form), and it does already have the playBytes response code.

Yes, but depending on which version of snips is used, you also need playBytesStreaming

[flatsiedatsie]

But isn't playBytesStreaming only required to play audio files? Voco doesn't (currently) stream audio to the satellites. At the moment, the satelites (Pi Zero's) generate the speech themselves. Altough if I could get ESP32 Rhasspy Satellite to work, then adding support for this would be worth it. A satellite the size of a sugar cube is amazing.

You mentioned that it would be necessary to play the audio files that indicate the wake word was heard. When I disabled the audio response, it still wouldn't work.

@flatsiedatsie can you check 7.3 if you still see double creations of the I2Stask?

My code already has a similar "only start the task once" check, and since then it doesn't load multiple times.

did you happen to try an installation of Voco yourself? I was hoping you'd have a look. You could probably more easily spot and fix the issue than I could.

[Romkabouter]

Yes, that is only required for playing audio.
I have tried voco indeed (see my comments in the Arduino version issue)
When voco completely implements the hermes protocol, I am convinced this code will work as a satellite for it :)
Voco basically uses snips, but snips' dialogue manager waits for a finished message as fas as I can recall.
That does not arrive (because it is not sent) and therefore times out. I have not investigated any further

[flatsiedatsie]

I have noticed something: wifi often crashes at boot. Currently the I2STask is created in the WifiDisconnected class. Perhaps that should wait until the wifi is connected? Otherwise streaming will attempt to start while there is no wifi yet?

[Romkabouter]

The WifiDisconnected is the initial state of the statemachine. That is why it is created there.
You could indeed create it in another state, but streaming of playing is not done when STREAM or PLAY is not set.
The task does nothing but loop in that case.

[flatsiedatsie]

Hmm, then why did it crash the wifi?

I got a bit further: I've managed to get "Hey Snips" detected! But only straight after booting the ESP32.

14:28:15.397 -> clientID = atomechoAudio
14:28:15.397 -> past asyncclient.connect
14:28:15.433 -> Wifi ready
14:28:15.433 -> end of setuppast audioServer connect, reached end of mqttdisconnected init
14:28:15.579 -> Enter MQTTConnected
14:28:15.579 -> Connected as atomecho
14:28:15.617 -> subscribed to topics
14:28:15.617 -> Enter Idle
14:28:15.617 -> Still Idle
14:28:15.617 -> idle started streaming
14:28:15.617 -> idle init complete
14:28:18.947 -> mqtt message received: hermes/hotword/azrxidia/detected
14:28:18.947 -> {"siteId":"atomecho","modelId":"hey_snips","modelVersion":"workflow-hey_snips_subww_feedback_10seeds-2018_12_04T12_13_05_evaluated_model_0002","modelType":"universal","currentSensitivity":0.5,"detectionSignalMs":-10,"endSignalMs":-10}
14:28:20.261 -> mqtt message received: hermes/hotword/toggleOff
14:28:20.261 -> {"siteId":"atomecho","sessionId":"689cb8cf-ee24-46e8-9fcd-be72fb175aab"}⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮ﾭ⸮xV⸮⸮⸮⸮⸮�⸮?T�⸮?⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮⸮ﾭ⸮xV⸮⸮⸮>⸮⸮�⸮?T�⸮?E⸮7⸮7⸮<⸮=⸮4⸮:⸮=⸮<⸮I⸮5⸮D⸮0⸮:⸮7⸮@⸮9⸮9⸮?⸮>⸮E⸮=⸮>⸮?⸮7⸮;⸮<⸮9⸮>⸮1⸮D⸮5⸮B⸮6⸮B⸮8⸮D⸮?⸮E⸮E⸮5⸮@⸮9⸮F⸮1⸮@⸮1⸮D⸮1⸮>⸮7⸮@⸮8⸮@⸮;⸮2⸮B⸮7⸮@⸮5⸮?⸮2⸮>⸮@⸮D⸮9⸮?⸮A⸮A⸮1⸮7⸮G⸮6⸮<⸮0⸮?⸮6⸮D⸮3⸮C⸮7⸮D⸮2⸮F⸮1⸮?⸮8⸮9⸮5⸮>⸮@⸮G⸮9⸮B⸮-⸮@⸮.⸮9⸮5⸮<⸮<⸮3⸮?⸮:⸮D⸮5⸮=⸮7⸮8⸮5⸮<⸮7⸮<⸮3⸮7⸮:⸮<⸮3⸮<⸮9⸮;⸮=⸮1⸮=⸮9⸮,⸮2⸮2⸮:⸮/⸮6⸮/⸮=⸮1⸮3⸮2⸮<⸮0⸮+⸮0⸮4⸮4⸮0⸮2⸮6⸮5⸮2⸮1⸮/⸮,⸮:⸮2⸮7⸮4⸮4⸮4⸮*⸮*⸮0⸮<⸮.⸮0⸮&⸮3⸮.⸮+⸮&⸮)⸮5⸮)⸮5⸮ﾭ⸮xV⸮⸮⸮⸮⸮�⸮?T�⸮?
14:28:20.373 -> got toggleOff
14:28:20.373 -> -its for us
14:28:20.373 -> idle react to HotwordDetectedEvent
14:28:20.373 -> Entry into HotwordDetected class
14:28:20.462 -> HotwordDetected class init complete
14:28:34.451 -> mqtt message received: hermes/hotword/toggleOn
14:28:34.451 -> {"siteId":"atomecho","sessionId":null}
14:28:34.451 -> got toggleOn
14:28:34.451 -> -toggleOn for me
14:28:34.451 -> Hotword switching to idle
14:28:34.451 -> Enter Idle
14:28:34.451 -> Still Idle
14:28:34.667 -> idle started streaming
14:28:34.667 -> idle init complete

Some strange things:

• mqtt message received: hermes/hotword/azrxidia/detected

That says azrxidia instead of atomecho. As if Snips is giving its own name to the site. On the next line the SideID is correct though.

• I wonder what all the gibberish in the output is about.

The HotwordDetected class disables and then re-enables streaming. Could it be that that is causing the voice activity being seen as down right when it's necessary ot hav it?

In Snips Watch:

[14:28:18] [Asr] was asked to listen on site atomecho
[14:28:20] [VoiceActivity] Down on site atomecho
[14:28:34] [Dialogue] session with id '689cb8cf-ee24-46e8-9fcd-be72fb175aab' was ended on site atomecho. The session was ended because one of the component didn't respond in a timely manner

The hotwordDetected init:

class HotwordDetected : public StateMachine
{
  void entry(void) override {
    Serial.println("Entry into HotwordDetected class");
    xEventGroupClearBits(audioGroup, PLAY);
    xEventGroupClearBits(audioGroup, STREAM);
    //device->updateBrightness(config.hotword_brightness);
    if (xSemaphoreTake(wbSemaphore, (TickType_t)10000) == pdTRUE) {
      //device->updateColors(COLORS_HOTWORD);
    }
    xSemaphoreGive(wbSemaphore);
    initHeader(device->readSize, device->width, device->rate);
    xEventGroupSetBits(audioGroup, STREAM);
    Serial.println("HotwordDetected class init complete");
  }

If I press the button to start a session, then things look a little different in Snips Watch:

[14:38:16] [Dialogue] session with id '5d60928f-dd37-4981-b124-56f2384626db' was started on site atomecho
[14:38:16] [Asr] was asked to listen on site atomecho
[14:38:18] [VoiceActivity] Up on site atomecho
[14:38:21] [VoiceActivity] Down on site atomecho
[14:38:23] [VoiceActivity] Up on site atomecho
[14:38:25] [VoiceActivity] Down on site atomecho
[14:38:27] [VoiceActivity] Up on site atomecho
[14:38:29] [VoiceActivity] Down on site atomecho
[14:38:32] [Dialogue] session with id '5d60928f-dd37-4981-b124-56f2384626db' was ended on site atomecho. The session was ended because one of the component didn't respond in a timely manner
[14:38:32] [Asr] was asked to stop listening on site atomecho

Here is seem that voice activity is up.. but it's perhaps not recognised as speech by the ASR?

[Romkabouter]

Can you please create a new defect for this? The original defect is something else
The task does not crash the wifi, did you use 7.4?

[flatsiedatsie]

It seems we were on the same track with the need for a delay to start the I2STask.

I'll open a new issue, good call.