Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alexa device discovery now broke #1243

Closed
smartypantsuk opened this issue Nov 2, 2020 · 149 comments · Fixed by #1266
Closed

Alexa device discovery now broke #1243

smartypantsuk opened this issue Nov 2, 2020 · 149 comments · Fixed by #1266
Assignees
Labels
Milestone

Comments

@smartypantsuk
Copy link

smartypantsuk commented Nov 2, 2020

Hi,

I have been using v5.3.1RC1 for a while, everything has worked well and the 'new device overwrites old device' bug has been fixed.
Sadly, and strangely, all my Alexa devices will no longer discover new bridge devices. After much diagnosing I have determined it to be a compatibility issue, possibly caused by an Amazon device update?

Testing:
Any v5.x.x version of the bridge does not allow for discovery. The logs show search and response between Alexa device and bridge but nothing is ever found on the Alexa device.

For reference I'm using ha-bridge on Windows 10.

Thanks

@luciano968
Copy link

I think the issue resides with Amazon pushing some sort of update.

@smartypantsuk
Copy link
Author

I think the issue resides with Amazon pushing some sort of update.

Yeah. I don't mind re-creating my devices in v4.x.x but when I use v4 I get the bug whereby I can't add new devices as it overides the older devices

@Juppers
Copy link

Juppers commented Nov 2, 2020

I just tried using 4.5.6 and devices were still not discovered.

@ggodart
Copy link

ggodart commented Nov 2, 2020

I have same problem with a completely new installation 5.3.0 and 5.3.1rc1 - never used HA-Bridge before. I thought it may be an echo problem so reset it to factory defaults before testing but devices that are correctly defined in HA-Bridge are not discovered by Alexa app.
UPnP trace shows comms between Echo and HA-Bridge and sending templates 1, 2 and 3 but Alexa App just reports No Devices found.
The guys in Misterhouse have the same problem and suspect a change at Amazon and spoke of something to do with long responses and gzip (beyond my comprehension)

@smartypantsuk
Copy link
Author

I just tried using 4.5.6 and devices were still not discovered.

V4.x.x always worked well for me. Just had too many other bugs. Maybe it's some other issue for you. Firewall? Different subnet, port?

@ggodart
Copy link

ggodart commented Nov 2, 2020

Echo and HA-Bridge connected to same router/subnet, and I can see udp traffic between them so unlikely to be firewall.

@smartypantsuk
Copy link
Author

I have same problem with a completely new installation 5.3.0 and 5.3.1rc1 - never used HA-Bridge before. I thought it may be an echo problem so reset it to factory defaults before testing but devices that are correctly defined in HA-Bridge are not discovered by Alexa app.
UPnP trace shows comms between Echo and HA-Bridge and sending templates 1, 2 and 3 but Alexa App just reports No Devices found.
The guys in Misterhouse have the same problem and suspect a change at Amazon and spoke of something to do with long responses and gzip (beyond my comprehension)

Did they find a fix, is there a link? There is an option to increase a timeout but perhaps not the correct thing. We could try increasing the timeout to 2000 or 3000

@ggodart
Copy link

ggodart commented Nov 2, 2020

No sign of a fix yet sadly

@smartypantsuk
Copy link
Author

No sign of a fix yet sadly

Still, v4 works for me. How odd

@phyllox
Copy link

phyllox commented Nov 3, 2020

Hello
For me, no device can be added/updated since one week. I changed nothing between the time it was working and the time it started not working. Probably an Alexa update. Called them, but they even don'y understand my questions.
Here is what I tried, but nothing worked :

  • Delelet all Alexa devices and make a new search. But Alexa re-decovered the same as before - even if deleted, renamed, even after I put HA-Bridge down, Crazy !
  • Changed devices numbering in HA-Bridge with different sequences (10x, 40x, 60x etc...)
  • Reset all my Alexa equipments (2xEcho, 1 spot, 1 Dot). After this and also after I also removed all rooms in Alexa, nothing was dicovered the HA-Bridge is up or down.
  • Changed the HA-Bridge version. restored a previous version from an image (it is a Synology virtual machine), changed version (which was 5.2.1) to 5.3.1RC1.
  • Deleted everything from HA-Bridge and recreated some test equipments.
  • Tried several config options in HA-Brige.
    Nothing of all that worked. Alexa is totally ignorig my HA-Bridge devices.
    Any thought ?
    Thanks

@ggodart
Copy link

ggodart commented Nov 3, 2020

I also tried HA-Bridge 4.5.6 with a clean install, created a device but still discovery didn't work. Tried with 2 versions of echo dot and a full sized echo.
I'm pretty sure Amazon has changed the interface/protocol and nixed all third party Hue emulations.

@roadsnail
Copy link

Also seeing the same issue (adding new devices) with 5.3.1RC1

Foolishly deleted all my HA-Bridge devices from amazon smart home devices, thinking they'd all return. But nothing detected.

Tried re-installing Gen2 Dot, Gen 3 Dot and Show 8 then re-discovering. No luck.

Then I saw this post regarding V4.x.x, so installed V4.5.6 deleted all in ha-bridge/data directory, added a few dimmable lights but was still unable to add new devices. That was last night. Just for the hell of it ran another discover earlier today, and 3 lights are detected and controllable okay, and one was detected but duplicated. I think one of the duplicates might have been working but I managed to delete the wrong duplicate! Doh! Tried to add the deleted duplicate to no avail.

I then restored V5.3.1RC1 using the /data db and config files as created by 4.5.6, and my 3 working lights are controllable by Alexa. I then tried adding my previously duplicated light, (still with V5.3.1RC1) but it is still not discovered. Currently sticking with my 3 working lights until Amazon resolves this issue whenever that happens to be.

So I can confirm for the sake of smartypantsuk sanity that V4.5.6 sort of worked for me but only after leaving things for about 12 hours before performing a re-discovery. (Some sort of caching thing at the Amazon Alexa device back-end?)

@smartypantsuk
Copy link
Author

Also seeing the same issue (adding new devices) with 5.3.1RC1

Foolishly deleted all my HA-Bridge devices from amazon smart home devices, thinking they'd all return. But nothing detected.

Tried re-installing Gen2 Dot, Gen 3 Dot and Show 8 then re-discovering. No luck.

Then I saw this post regarding V4.x.x, so installed V4.5.6 deleted all in ha-bridge/data directory, added a few dimmable lights but was still unable to add new devices. That was last night. Just for the hell of it ran another discover earlier today, and 3 lights are detected and controllable okay, and one was detected but duplicated. I think one of the duplicates might have been working but I managed to delete the wrong duplicate! Doh! Tried to add the deleted duplicate to no avail.

I then restored V5.3.1RC1 using the /data db and config files as created by 4.5.6, and my 3 working lights are controllable by Alexa. I then tried adding my previously duplicated light, (still with V5.3.1RC1) but it is still not discovered. Currently sticking with my 3 working lights until Amazon resolves this issue whenever that happens to be.

So I can confirm for the sake of smartypantsuk sanity that V4.5.6 sort of worked for me but only after leaving things for about 12 hours before performing a re-discovery. (Some sort of caching thing at the Amazon Alexa device back-end?)

The funny thing was i tried v5.3.1.RC1 again last night and it found 2 new devices. Then it wouldn't find anything else after that.

However, you could be right about the caching issue because one of the devices that it found did not exist in the HA-bridge database. It was a test device that I created but then deleted probably a few hours previous. So where did this ghost device come from if not some cache.

@roadsnail
Copy link

@smartypantsuk - Re. caching... With regard to my duplicate light I mentioned above, I'm guessing one of them was discovered earlier (under 4.5.6) the other (which did not work from Alexa voice or the Alexa Android App) must be lurking in the cloud somewhere. I definitely reset all of my devices, which presumably clears any on-device stuff? Who knows?

It's interesting that Alexa discover via 5.3.1RC1 managed to pick up any devices. I have had zero success.

@ggodart
Copy link

ggodart commented Nov 3, 2020

The guys over at Misterhouse think Amazon have changed the protocol to not use gzip in the header requests during discovery. They are going to do some more testing. It seems devices already discovered may continue to work, but once deleted you can only discover them again if they are in a cache somewhere. I think this is going to need someone from HA-Bridge to snoop the network with a real Hue device to see what should be going on and then patch HA-Bridge accordingly, I don't see Amazon helping here unless they cannot discover real Hue devices themselves now.

@smartypantsuk
Copy link
Author

Seems very odd. I have some hue stuff here and a signed sniffer but I've decommissioned everything at present. Really like the hue lights but not zigbee in general. It's a shame they didn't go zwave with the hue system.

I digress. Is ha-bridge even developed anymore?

@ggodart
Copy link

ggodart commented Nov 3, 2020

most recent release was 26 Mar

@smartypantsuk
Copy link
Author

@bwssytems Is this still developed?

@Perforex
Copy link

Perforex commented Nov 3, 2020

I have the same problem with 5.3.0 - Added a new device yesterday and when it wasn't discovered I foolishly removed all devices in the hope that would clear the problem. Alas it only made it worse, now Alexa refuses to find any of my devices that are defined in HA-Bridge.

After several discoveries, similar to someone else who commented, one of my devices re-appeared but nothing in it's definition is in anyway unique . Here's hoping for a solution.

@smartypantsuk smartypantsuk changed the title Alexa device discovery broke in v5.x.x but NOT in v4.x.x Alexa device discovery now broke Nov 3, 2020
@smartypantsuk
Copy link
Author

Okay I've edited the title to better reflect the problem. After further testing it appears that No versions are working with respect to discovery.
I tried tonight again with v4 and v5 I now have no devices in Alexa actually responding and discovery doesn't work at all.

@jens7841
Copy link

jens7841 commented Nov 3, 2020

I can confirm that issue.
I build my own Hue-Emulator for an Arduino/ESP-8266 and the Problem is the same. Alexa stoppt discovering the Bridge / the lights...
Alexa still find it by UDP/SSDP and calls all URLs correctly but dont show Up the lights...
I tried all i know but nothing worked ...

I can only guess, maybe Amazon wants to force us to write an Alexa Skill for that?
Some official answers from Amazon would be nice... :D

@audiofreak9
Copy link
Collaborator

I have never had discovery issues very long time user of HA-Bridge, even swapped out routers a year ago without issue (Verizon to Asus). I tried adding a new test device and my echos are able to discover the device. (Gen 1 Echo, multiple Gen 1 & 2 Dots, Gen 1 & 10.1" Premium Show, Gen 1 Spot)

However the alexa smart home page is surely going through changes, the page will change for me to one I've never seen before following these steps (but not every time):

  • bottom of the page click discovery
  • when discovery is complete refresh the page (F5 or click refresh on the browser)
  • the URL glitches and changes to #cards without content, just a "More" button
  • click Smart Home from the navigation, this displays the following all on one page: my Device Groups, my enabled Smart Home skills, a single box describing how many discovered smart devices I have currently (count is way off, says 196 I have 115) and includes a link to discovery, a box with a list of my devices, a box with a list of my scenes, then at the bottom a red link to delete all devices and groups.

Anyone else seeing this combined page?

@smartypantsuk
Copy link
Author

I have never had discovery issues very long time user of HA-Bridge, even swapped out routers a year ago without issue (Verizon to Asus). I tried adding a new test device and my echos are able to discover the device. (Gen 1 Echo, multiple Gen 1 & 2 Dots, Gen 1 & 10.1" Premium Show, Gen 1 Spot)

However the alexa smart home page is surely going through changes, the page will change for me to one I've never seen before following these steps (but not every time):

  • bottom of the page click discovery
  • when discovery is complete refresh the page (F5 or click refresh on the browser)
  • the URL glitches and changes to #cards without content, just a "More" button
  • click Smart Home from the navigation, this displays the following all on one page: my Device Groups, my enabled Smart Home skills, a single box describing how many discovered smart devices I have currently (count is way off, says 196 I have 115) and includes a link to discovery, a box with a list of my devices, a box with a list of my scenes, then at the bottom a red link to delete all devices and groups.

Anyone else seeing this combined page?

Can you confirm the bridge version you are using and try creating a new test device to see if that discovers as of now?

@smartypantsuk
Copy link
Author

On an associated note, these changes by Amazon may be related to a development brief I read a few months back. Specifically, that coming soon would be the ability to use some smart home skills offline! In other words, some skills for controlling your smart home will not require to be online. This would be useful and may allow a custom skill to work similar to ha-bridge.

But maybe not. Time will tell.

@amyth1884
Copy link

After deleting devices from here https://www.amazon.in/hz/mycd/myx#/home/alexaPrivacy/deviceHistory, I was able to discover devices but with delays, patience needed, as highlighted somewhere in the thread it takes some time to clear cached devices, to be on safe side I did a 20 sec reboot for the Echo Plus.

@phyllox
Copy link

phyllox commented Nov 4, 2020

Hello
Glad for people who can make it working. For me, whatever I try, still the same, no discovery after a lot of tries.
Have to say that I have also a real Hue bridge, and that devices (mainly bulbs) on this Hue bridge are working as before. But devices on Ha-Bridge (power switchs, power plugs...) are not discovered. Real Hue bridge is on canal 20. Could this canal setting be a problem for HA bridge where I did not find a way to configure it ? I have several Zigbee gateways using a different canal (Hue, Xiaomi,, plus 2x Zigbee dongles).
Also not clear for me if we are just discussing here between HA-Bridhe users, or if somebody in charge of HA-Bridge software is there - if the softwre is still maintained !?
Can the people that have it working say ig they just have HA-Bridge or also a Hue bridge, and maybe send a screenshot of the HA-bridge config ?
Thanks

@sanderlv
Copy link

sanderlv commented Nov 4, 2020

Ok, so I added a new device in my habridge to day (5.3.0) and I stumbled on the fact it is not added (discovered) by alexa. "old/existing" devices do work but no new discovered. And then I stumbled on this topic... But I am not seeing a solution yet, correct?

@ggodart
Copy link

ggodart commented Nov 4, 2020

Correct, I've emailed [email protected] and [email protected] and got no reply.
Its safe to say that there is no solution to this yet.

@sanderlv
Copy link

sanderlv commented Nov 4, 2020

So this means this ENTIRE project is not functional anymore!? That would be the worst news before christmas after covid :-).

@bwssytems can you please advise?

@davidohne
Copy link

I didn't want to recreate all the devices so I edited the devices.db
You have to change and prolong the uniqueid of every device (replace all helps..).
The new uniqueid had to be longer than my old ones. This is an example of a device which gets recognized again:
00:11:22:33:44:55:66:64-64
If the last to digits are not the same than the two digits before (64 to 64 in this case) the device will not be recognized. There were some devices where the last digits were just 00 like 00:11:22:33:44:55:66:64-00 -> this will not work. It has to be 00:11:22:33:44:55:66:64-64.

@CatmanV2
Copy link

Thanks to everyone involved in this fix. Spent an age chasing multicast traffic across my network. Found this thread, updated in about 45 seconds and all sorted.

PHEW

C

@tbclark3
Copy link

tbclark3 commented Nov 24, 2020

I gave up on using habridge directly with Alexa several months ago. One by one the devices "stopped responding" and usually couldn't be re-discovered even if deleted. What I found, however, is that Smartthings can readily discover habridge devices and then Alexa discovers them from Smartthings within seconds. It means an extra step in that Alexa talks to my Smartthings hub, then the Smartthings hub talks to the habridge, but the delay is not noticeable. Everything just works including dimming.

Interestingly, the Smartthings server takes the habridge device unique ID, eg uniqueId: 00:17:88:5E:D3:C8-00, and converts it to the form 02420A010A40/200 where the base number (before the slash) is the ID that Smartthings assigned to the habridge and 200 is the device ID. All of the child devices of the habridge share the same base id number. It advertises the device through the Smartthings Alexa app and Alexa discovers it immediately and without being asked to discover devices.

@CatmanV2
Copy link

Interesting. My HA Bridge has been pretty much rock solid. Where I have had 'Device is not responding' it's where I've managed to get duplicates.
The discovery is a bit of a PITA, but I don't change stuff often enough to worry about.
However I have no experience using it outside of openLuup. Alexa and Vera was atrocious (well Vera and anything, really)

C

@K1WIZ
Copy link

K1WIZ commented Nov 26, 2020

This worked for me (the RC2 update) I had to delete my "data" directory after updating my .jar to the RC2 version and then recreate all my devices. A pain, but it worked. Thank you to all who have (and continue to) contribute to this essential project. You all saved me from the wrath of my wife who was giving me the evil eye because the bridge was broken :)

@Reggs01925
Copy link

I've used HA Bridge with Homeseer under windows 10 for years. I've always run it on port 85 until it stopped working a few days ago. Wouldn't discover any devices and therefore lost echo control of approx 45 events and devices!
I updated to the latest jar and tried changing device ID's as outlined above but nothing worked. Then I read a post on the Homeseer forums where a member posted that HA Bridge only ran on port 80 which I knew was incorrect. Nevertheless, I changed my setup to port 80 and all devices were suddenly rediscovered and the bridge is working great once again. Has there been a change that stops HA Bridge running on anything other than port 80 under windows 10? It seems odd that this would suddenly happen.

@smartypantsuk
Copy link
Author

Then I read a post on the Homeseer forums where a member posted that HA Bridge only ran on port 80 which I knew was incorrect"

Ha-bridge works on whatever port you specify, Alexa, however, doesn't. Amazon changed the discovery behaviour some time ago so that it only discovers on port 80.

@CatmanV2
Copy link

Pretty sure that's not quite right. AFAIK Alexa is using UPNP so port number is not really an applicable concept. However, I can categorically state that my HA bridge is running on 8080 (and UPNP response 5000) and with the new .jar discovery was working fine this week. Of course, it might have changed since Tuesday.
It does look as though there's been another upgrade (but trying to remember versions is probably not going to work) so if discovery as suddenly stopped working (again) there might be an issue

@smartypantsuk
Copy link
Author

Pretty sure that's not quite right. AFAIK Alexa is using UPNP so port number is not really an applicable concept. However, I can categorically state that my HA bridge is running on 8080 (and UPNP response 5000) and with the new .jar discovery was working fine this week. Of course, it might have changed since Tuesday.
It does look as though there's been another upgrade (but trying to remember versions is probably not going to work) so if discovery as suddenly stopped working (again) there might be an issue

There are some oddities that can't be quite explained regarding Alexa discovery. What works for some doesn't for others, however, port 80 became the necessary change many months ago.
Discovery stopped working for many and using port 80 fixed it. Maybe it's a hue requirement and not Alexa, but Google home also required the bridge to run on port 80.

@CatmanV2
Copy link

CatmanV2 commented Nov 29, 2020 via email

@bwssytems
Copy link
Owner

So, for discovery to now work, it seems you need to delete all devices in the alexa echo app or home page. Make sure you are using 5.3.1RC2 and renumber your devices with a new seed in the bridge control tab. Then you can re-discover devices.

@CatmanV2
Copy link

CatmanV2 commented Nov 30, 2020 via email

@bwssytems bwssytems added this to the 5.3.1 milestone Dec 2, 2020
@bwssytems
Copy link
Owner

RC3 is now out in the releases section. Updated upnp handling and device unique ids to be more specific. Also it seems the unique ids have been too long since inception and that may be the root cause. Delete all your devices in alexa and use renumber before you discover.

bwssytems added a commit that referenced this issue Dec 14, 2020
Dev branch 5.3.x to create v5.4.0

#1044 Alexa app, error "Device doesn't support requested value" bug question
#1118 Device status update in Alexa App is not working correctly for ha-bridge devices bug duplicate question
#1126 Can't get XY ColorData into URL enhancement question
#1127 Can't get separate hue, saturation and brightness enhancement question
#1128 Wrong color calucation in HS mode bug question
#1131 Changing color temperatur is not working correctly bug question
#1132 Conversion from CIE to RGB incorrect bug question
#1139 Device ID creation fails bug question
#1150 Home Assistant gets null exception during initialization bug question
#1170 Alexa not sending the correct color codes duplicate
#1171 Please add option: If bright request present with on, only send dim enhancement
#1184 Alexa will not discover HABridge question
#1191 When "Use UPNP Address Interface Only" is selected, echo cannot find ha-bridge bug
#1210 [Enahnacement] Have link button timeout be configurable enhancement
#1215 Alexa unable to discover recently added devices to ha-bridge duplicate question
#1227 Feature request: ${intensity.previous} value passing control enhancement
#1239 HA-Bridge has suddenly given up itself - Exception in thread "main" java.lang.NullPointerException bug question
#1243 Alexa device discovery now broke question
#1253 HABridge hue ColorDecode 0 0 0 bug question
#1256 Homeassistant devices not appearing bug question
This was referenced Dec 21, 2020
@npaisnel
Copy link

Interesting. My HA Bridge has been pretty much rock solid. Where I have had 'Device is not responding' it's where I've managed to get duplicates.
The discovery is a bit of a PITA, but I don't change stuff often enough to worry about.
However I have no experience using it outside of openLuup. Alexa and Vera was atrocious (well Vera and anything, really)

C

I have had this "Device is not responding" comment from Alexa when I have some how managed to get an entry in the "Dim item" line and I am trying to switch a heyu appliance module with no Dim possibility.

The Entry in the Dim item line was simply an empty pair of square brackets... []

Removing that erroneous Dim Item and the Device is Not Responding response went away.

@CatmanV2
Copy link

CatmanV2 commented Apr 25, 2021 via email

@npaisnel
Copy link

I did notice that even the new ID's have the -XX at the end and not the :XX yet it all still seems to work with the dash.

@jackwan1
Copy link

jackwan1 commented Jul 4, 2021

I am new here.
Can someone summarize the result of this discussion? for all I can see, the HAbridge must be running on port 80 to make the Alexa discover devices defined in HAbridge. I cant run ha on port 80 because it is occupied on my server.

@smartypantsuk
Copy link
Author

smartypantsuk commented Jul 4, 2021

I am new here.
Can someone summarize the result of this discussion? for all I can see, the HAbridge must be running on port 80 to make the Alexa discover devices defined in HAbridge. I cant run ha on port 80 because it is occupied on my server.

Amazon Alexa devices will only look for devices on port 80. Simple as that. The few discrepancies from posters on here who say their systems are working on an alternative port must have 1 or more Alexa devices either stuck on an old software version or the device hasn't updated properly, which may be yet another kind of caching issue, which seem to plague Alexa devices.

You could of course setup reverse proxy. Some examples are on main page

@jackwan1
Copy link

jackwan1 commented Jul 9, 2021

Thank you.
I finally got Domoticz working with Alexa on port 443 and ddns, however, it is a common port and if you open the port on your router, it is subject to hacking. You need to set SSL for the port/router.

I am working on reverse proxy server, actually, nginx IS the process holding port 80 on my Ubuntu server, should be easy to do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.