Home assistant hyperion integration is broken

[nurikk]

Hi, I can't add hyperhdr to home assistant using default integration.

Bug report, debug log and your config file (FULL LOGS ARE MANDATORY)

HyperHDR logs:

2021-09-09T15:28:24.830Z [JSONCLIENTCONNECTION] (JsonAPI.cpp:89) Client '::ffff:192.168.1.200' switch to HyperHDR instance 0
2021-09-09T15:28:24.835Z [JSONSERVER] (JsonServer.cpp:105) New connection from: ::ffff:192.168.1.130 
2021-09-09T15:28:24.837Z [JSONCLIENTCONNECTION] (JsonAPI.cpp:89) Client '::ffff:192.168.1.200' switch to HyperHDR instance 0
2021-09-09T15:28:24.970Z [JSONSERVER] (JsonServer.cpp:121) Connection closed
2021-09-09T15:28:24.976Z [JSONSERVER] (JsonServer.cpp:121) Connection closed

Home assistent logs are empty, ui shows me this message

Steps to reproduce

1. Navigate to homeassistant -> Configuration -> Integrations -> Add integration
2. Search hyperion
3. Enter hyperion
4. Select hyperion
5. Enter ip addess
6. Click ok

What is expected?

Add hyperhdr integration

What is actually happening?

Nothing, only error message

System

HyperHDR Server: 
- Build:           (HEAD detached at 4e4031c) (Awawa-eba222b/4e4031c-1630846534)
- Build time:      Sep  5 2021 13:29:15
- Git Remote:      https://github.com/awawa-dev/HyperHDR
- Version:         17.0.0.0beta2
- UI Lang:         auto (BrowserLang: en-GB)
- UI Access:       default
- Avail Capt:      Linux (V4L2)
- Database:        read/write

HyperHDR Server OS: 
- Distribution:   Raspbian GNU/Linux 10 (buster)
- Architecture:   arm
- CPU Model:      ARMv7 Processor rev 3 (v7l)
- CPU Type:       Raspberry Pi 4 Model B Rev 1.1
- CPU Revision:   c03111
- CPU Hardware:   BCM2711
- Kernel:         linux (5.10.17-v7l+ (WS: 32))
- Qt Version:     5.11.3
- Browser:        Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36

Home assistant

System Health

version	core-2021.9.5
installation_type	Home Assistant Supervised
dev	false
hassio	true
docker	true
user	root
virtualenv	false
python_version	3.9.6
os_name	Linux
os_version	4.9.241-77
arch	aarch64
timezone	Asia/Singapore

Home Assistant Community Store

GitHub API	ok
Github API Calls Remaining	4797
Installed Version	1.15.2
Stage	running
Available Repositories	881
Installed Repositories	12

Home Assistant Cloud

logged_in	false
can_reach_cert_server	ok
can_reach_cloud_auth	ok
can_reach_cloud	ok

Home Assistant Supervisor

host_os	Ubuntu 18.04.5 LTS
update_channel	stable
supervisor_version	supervisor-2021.09.0
docker_version	20.10.7
disk_total	29.1 GB
disk_used	8.7 GB
healthy	true
supported	failed to load: Unsupported
supervisor_api	ok
version_api	ok
installed_addons	Duck DNS (1.12.4), ESPHome (2021.8.2), FTP (4.2.1), File editor (5.3.3), Git pull (7.12.2), Glances (0.13.0), Log Viewer (0.11.1), Mosquitto broker (6.0.1), NGINX Home Assistant SSL proxy (3.0.1), Node-RED (10.0.0), Portainer (2.0.0), SSH & Web Terminal (9.0.1), WireGuard (0.5.1)

Lovelace

dashboards	1
resources	5
views	3
mode	yaml

[awawa-dev]

SSDP has been reworked in v17 and may lead to incompatibility with some old Hyperion plugins.
Log from Home Assistant is misleading: says it can't no find ATTR_UPNP_SERIAL = "serialNumber" (probably this: link 'no_id') , but if you type your HyperHDR SSDP config address it should be present at http://HYPERHDR_IP:PORT/description.xml
Unless there is some workaround without returning back to the old stuff in HyperHDR then maybe it could be 'fixed'. Other and proper way is to create a plugin for HyperHDR as some other things in its JSON api also has changed.
Hope you understand since this is an external application using a plugin created for Hyperion NG, can't consider it as a bug in HyperHDR.

[nurikk]

Oh, that's sad that we've lost api compatibility with existing HA integration. HA integration requires freaking hell of copy-paste work. Anyway, thanks for your great work!

[thehijacker]

Hello.

Home Assistant integration was working fine with 16.x release but not it is completely broken. I had automation that turned on LED device and USB capture when my TV went on, but this is not working now. Can I use some sort of API call (http rest?) to enable/disable this?

Thank you.

[awawa-dev]

Hi
It's not broken but no longer compatible with Hyperion NG plugin. Still you can use HyperHDR's JSON API (now it is even extended with new features compared to Hyperion one) to send commands. For most of them you can use new JSON wizard ('advanced' tab) to create them for you: probably link 'All instances (Global)' could work here.

[thehijacker]

Managed to implement this json_rpc using HA rest call. FYI:

rest_commands:
  hyperhdr_usb_capture:
    url: http://192.168.28.64:8090/json-rpc
    method: POST
    headers:
      accept: 'application/json, text/html'
      user-agent: 'Mozilla/5.0 {{ useragent }}'
    payload: '{	"command":"componentstate", "componentstate": { "component":"VIDEOGRABBER", "state": {{ state }} } }'
    content_type:  'application/json; charset=utf-8'
  hyperhdr_led_device:
    url: http://192.168.28.64:8090/json-rpc
    method: POST
    headers:
      accept: 'application/json, text/html'
      user-agent: 'Mozilla/5.0 {{ useragent }}'
    payload: '{	"command":"componentstate", "componentstate": { "component":"LEDDEVICE", "state": {{ state }} } }'
    content_type:  'application/json; charset=utf-8'

And then in my automation:

automation:
  - alias: 'TV power over 60W turn on Ambilight'
    id: 'tv_power_over_60W_turn_on_ambilight'
    trigger:
      - platform: template
        value_template: >-
          {{ states('sensor.blitz_socket_sony_energy_power') | int > 60 and is_state('media_player.chromecast_audio', 'off') }}
    action:
      # Prižgi USB capture
      - service: rest_command.hyperhdr_usb_capture
        data: {state: "true"}
      # Prižgi LED okrog TV
      - service: rest_command.hyperhdr_led_device
        data: {state: "true"}

Now to test how is CPU usage on my RPi Zero W. It was just under 100% when USB grabber was on. I think I need better Raspberry Pi for running HyperHDR.

[awawa-dev]

I think switching to GET request could make the configuration even simpler, but not using HA so thank you for providing example 👍

It can be done even for Rpi zero for one instance if your grabber has a scaler and its capturing is not limited to 1080p/MJPEG and high frame rate : everything is a matter of the proper configuration beginning with lower resolution, choosing YUV, a 'quarter of frame' mode and 'frame rate decimation' in the grabber, sparse processing and not setting too high values in the 'smoothing' component (or disable it completely): both processing tab.

[thehijacker]

My smoothing setting:

And USB capture settings:

Only smoothing and blackbar detection is enabled. When needed I enable USB capture and LED device. Had much better results with HyperHDR compared to Hyperion NG. Flickering is gone. Colors are more vivid. If you have any more optimisation advice I would be very grateful.

This Home Assistant issue was just a little setback. Need to figure our how to use effects. Winter is coming and I would like to have some light effects around the TV when we are not watching it. Maybe HA Hyperion integration will be updated to support also this latest version of HyperHDR.

Thank you.

[awawa-dev]

The problem with HA Hyperion NG plug-in is probably only configuration matter when it reads application configuration from SSDP file described above: some names have changed. You may try setting 'software frame skipping' to 2, but it will reduce your framerate to around 10FPS. But first check in logs 'av. delay' for processing frame: they are refresh every one minute. For smoothing refresh time set it to 50-150ms. it does not effect the performance but only delay. And probably you want to enable 'antiflickering timeout' or at least test it 😉

[mjoshd]

For what it's worth, I set up a new sdcard (rpi4, aarch64) and only had to uncheck "Classic HyperHDR calibration" on the 'Image Processing' & 'Remote Control' pages to get back the standard 'Brightness' percentage (instead of 'Luminance') and now it seems to work perfectly well with the existing HA Hyperion NG integration.

Is there any down-side to disabling "Classic" calibration?

I had to implement my own toggle for enabling/disabling HDR Tone Mapping but that was pretty easy thanks to the new JSON API playground.

Perhaps some day someone will consider making a proper HyperHDR integration for HA... who knows, Hacktoberfest is coming.

HyperHDR is really amazing and blows Hyperion out of the water!

Thanks! @awawa-dev

[awawa-dev]

Thanks @mjoshd :) Both calibrations come from Hyperion (old & new). I just remixed them because I don't like new calibration and find the old one much easier, but I want to give a user a choice. It's offtopic but from my experience calibration is useful for tuning up brightness or if the wall behind is not white and we want to balance it. In other cases the effect is limited as you calibrating LED for full color: maximum red, blue green etc but the LED strip behaves differently on darker color and it's not linear. In that case 'classic' calibration saturation could help a bit. For RGBW cold white I use default settings without any modification but for that type of LED strip white channel algorithm is crucial.

[mjoshd]

@nurikk @thehijacker @awawa-dev

I've done some digging as to why my installation was working while others are unable to get it going and found a workaround until there is a proper HyperHDR HA integration. It was because I had Hyperion.ng set up and running with HA before I installed HyperHDR on my Pi, and I never removed the HA integration during the changeover.

I did some tests this morning and found an easy way for anyone to do the same.
Steps to replicate:

• log into your router and set a DHCP reservation, or static IP, for the Pi running HyperHDR
• power down your Pi and remove your HyperHDR SD card
• grab an extra SD card and install any version of Hyperion.ng (I performed this test with 2.0.0-alpha.10)
• set up the HA Hyperion Integration
• power down the Pi, remove the Hyperion SD card and reinstall your HyperHDR SD card
• in HyperHDR settings, uncheck "Classic HyperHDR calibration" on the 'Image Processing' & 'Remote Control' pages
• done!

With this workaround, the only thing really missing in HA is the ability to toggle HDR Tone Mapping/HDR Global which can easily be done through HA or Node-RED.

I think the root of the incompatibilty is that the HA integration has a requirement (in its manifest.json) which defines use of hyperion-py

In the hyperion-py library are some constants referring specifically to Hyperion and it looks like HyperHDR's description.xml now no longer has any references to hyperion (as of v17), which is probably for the best and also why it used to 'just work' with v16.

I plan to make a fork of the HA integration & hyperion-py library to try and see if I can adapt it and/or create a proper HyperHDR HA integration. I'm pretty familiar with coding but know nothing of python so it could take a while. Hopefully the workaround I described above will help others to get HyperHDR working with the existing HA Hyperion integration until then.

[malcolmcdixon]

Hi, I found a fairly easy way to resolve by modifying line 402 of config_flow.py in the usr/src/homeassistant/homeassistant/components/hyperion folder in my docker container:

Change:
hyperion_id = await hyperion_client.async_sysinfo_id()
To:
hyperion_id= "your_serial_no"

Serial no obtained from description.xml file as described above.

Until an update resolves this, it will need changed on every homeassistant installation update though.

I'd love to be able to do a HyperHDR integration but I'm just not that experienced, I don't understand most of the code tbh.

EDIT: I managed to understand it enough to edit the code to add the switch entity for HDR Mode though!

@awawa-dev Thanks for this latest version, managed to get HDR Mode to switch automatically when using Plex :-)

[tavalin]

Thanks for this latest version, managed to get HDR Mode to switch automatically when using Plex :-)

Slightly off topic but how did you do this?

[malcolmcdixon]

@tavalin Hi Daniel, I just used Node-RED, notably the node https://flows.nodered.org/node/node-red-contrib-plex-ws. It has only had limited testing but has worked so far in my case, I'm sure the flow could be improved upon.

Then use a filter node to block changes if msg.session.Media[0].videoProfile does not change. Next a switch node to route on whether msg.session.Media[0].videoProfile == main 10, or not. Finally, use http request nodes to send the commands to HyperHDR. I hope that helps, any issues just ask and if you find a way of improving please let me know.

[malcolmcdixon]

To all concerned, I've found the cause of the issue, it is due to the JSON returned from sysinfo command http://192.168.1.72:8090/json-rpc?request={"command":"sysinfo"}. The key returned for the id is info.hyperhdr.id whereas the HA integration relies on dermotduffy/hyperion-py which looks for info.hyperion.id as can be seen in the code at line 1471 in client.py:
sysinfo_id = ( sysinfo.get(const.KEY_INFO, {}) .get(const.KEY_HYPERION, {}) .get(const.KEY_ID, None)
and const.py defines KEY_HYPERION as:
KEY_HYPERION = "hyperion"

It would be great if someone more knowledgable than I could merge dermotduffy's code and the HA Hyperion integration into a new HyperHDR integration, I'd certainly help where I could.

[bverwijst]

If people just want a simple switch to turn on or off your lights with Alexa or in a Lovelace dashboard this is a simple solution:

Create this switch in your configuration:

switch:
- platform: template
    switches:
      hyperion:
        friendly_name: Hyperion LED
        turn_on:
          service: rest_command.ambi_on
          data:
            component: 'LEDDEVICE'
            state: 'true'
        turn_off:
          service: rest_command.ambi_on
          data:
            component: 'LEDDEVICE'
            state: 'false'
        icon_template: mdi:led-strip-variant

And this rest command:

rest_command:
  ambi_on:
    url: http://HYPERHDR_IP:PORT/json-rpc
    method: POST
    headers:
      accept: 'application/json, text/html'
      user-agent: 'Mozilla/5.0 {{ useragent }}'
    payload: '{	"command":"componentstate", "componentstate": { "component":"LEDDEVICE", "state": {{ state }} } }'
    content_type:  'application/json; charset=utf-8'

Thanks to @thehijacker for the rest command setup. Works like a charm like this.

[mjoshd]

@nurikk @tavalin @bverwijst @thehijacker @deimoszero @awawa-dev

...I plan to make a fork of the HA integration & hyperion-py library to try and see if I can adapt it and/or create a proper HyperHDR HA integration...

It took a long time but I finally had a chance to sit down and dig into the code.
Now there exists a proper HyperHDR custom component for Home Assistant and it even has a toggle for HDR Tone Mapping! 😁

I've tested it at my house and it works well so I thought I'd share!

This is my first time working with python and HA custom components so let me know if any parts of it don't work as expected by opening an issue on my repo; no need to clog up the issue tracker here.

Cheers! 🎉

[mjoshd]

[bverwijst]

Awesome work! Thanks, I just installed it here and it's working absolutely fine. Much easier to integrate this with my node-red automations too now. Might be worth posting it on the Home Assistant forum too.

[awawa-dev]

Great job @mjoshd ! Although I don't use HA I've managed to install it to test the plugin on my backup Rpi and seems it works fine for my basic configuration 👍 I will mention about your project in readme.md and in next release info.

[sorted01]

That's awesome! Very very cool! Thank you :)

[pOaW]

I'm getting an Error, after the token step of the integration installation. Any thoughts?

[bverwijst]

Can confirm, after the latest Home Assistant update my button or any automations with HyperHDR stopped working.