Fully support dual channel actors

[koalo]

In the FHEM module, a dual channel actor is represented as three devices.
One main device (e.g. 4376bb) and one for each channel with appended
channel number (e.g. 4376bb01 and 4376bb02).

For consistency, this should be also done for pyduofern, but
was only partly implemented, yet. In particular, these channel-specific
devices where not added as own devices and thus could not be addressed.

[gluap]

First of all: Thank you a lot for implementing the multi channel actor. I really appreciate it. I'm currently looking into why the changes break Travis (may take a few days).

One question while I'm still at it:
You mass-replaced code by hash in your pull request and modified the update_state function to work on a passed-in dict instead of going via self, basically making it a @classmethod. The purpose of this seems to be making comparisons with the FHEM code easier to find my mistake in porting the functionality, or am I overlooking something? I am asking because I'm thinking about how to make it more pythonic post-commit while preserving your added functionality.

[koalo]

Sorry, I overlooked the CI output and did not run the unit tests. 2dc3f6a somehow leads to a blocked loop in test_replay.py. I will have a look, too, when I have the time for it.

The mass replacement of code with hash has less to do with the comparison with the FHEM code, but is actually based on my assumption of the difference between code and hash when looking at this line:
https://github.com/gluap/pyduofern/blob/master/pyduofern/duofern.py#L259
My assumption was that hash contains the reference to the current module_definition (either module_definition01 or module_definition02, while the code itself is not channel specific. The fact that this makes update_state a @classmethod is just a result of that and I agree with your rating as quite "unpythonic".

Have you already had a look in the commit messages of the individual commits? They should also give some hints about the background. The main issue here is that we deal with different notions of what a "device" is. For Rademacher, a device is the physical unit that has the antenna, while for the FHEM module, a device can also be a single channel of a dual channel actor (see in particular the commit message of 2dc3f6a).

In my opinion, the cleanest solution would be to stay with the Rademacher notion of a device and only (de)multiplex the device to two switches just in the home assistant component, but since this would be quite different from the FHEM approach I thought it would be better to stay with this double notion.

If you have any ideas how this could be realized more smoothly and more pythonic, I am happy to change the pull request, but you can also apply them directly. Whatever you like. Also I am happy to go more into detail about my findings and can also provide real-world testing.

In any case, I am very happy, that thanks to your code, I was now able to switch from FHEM to HomeAssistant for my home automation. Thanks a lot!

[gluap]

I found out where the Travis issue comes from, the Duofern.set() method is broken (at least for rollershutters). I think I'll be able to look into this some more tomorrow.

[gluap]

The travis timeout happens because in this line hash is undefined (same issue with all other references to hash in this method). To merge upstream we have to fix this.

Unfortunately just going back to using code or device_id here does not help because update_state was refactored to expect the module_definition instead of a device id. It may make sense to leave this as is and instead revert to the previous behaviour of update_state and instead adapting the code.

[gluap]

Instead of modifying the passed dict here, it may make sense to just fix the code or device_id in Duofern.parse(). Therein one could introduce a device_id then every time hash is set to a new value one would have to modify device_id accordingly. Or (after reading your thoughts on device in FHEM vs device for Rademacher) one might make update_state subdevice-aware by coming up with a a subdevice id.

[gluap]

This rename makes a lot of sense, thank you!

[gluap]

I pulled in the upstream modification of the unit test to stop it from entering the infinite loop. That does not fix it but at least this way we can find out faster whether a fix works.

[koalo]

Good catch! Thanks!

After reading your comments, I realize I was not very careful with the renaming and we should put some more thought into the names. There are at least the following names we should refine and decide if they refer to a device with or without channel:

module_definition*
code
hash
device_id

I had a look into the Rademacher manual and they use "code" for the 6 digit identifier. And for one "code" there can be one or more "channels". So we have two options:

1.) Maintain the FHEM notion of a device and keep three devices per dual-channel actor in the duofern.json. Then we should consistently refer to the 8 digit identifier as device_id als I did at some places, but not consistently. For example, we should then use self.modules['by_device_id'][...] instead of self.modules['by_code'][...] and apply the name consistently at all the other places.

2.) Let a device refer to a device in the Rademacher sense. This would mean code and device_id is the same so we should drop device_id completely. And then we introduce a new channel parameter where it is necessary.

What is your opinion about that?

[gluap]

@kaolo thank you for your clear thoughts.

Firstly: At first I favored your 2 but now I think 1 may be the better way. I notice that after some thinking my considerations on suggestion 2 have become quite lengthy and 1 may be more straightforward to implement.

In the context of your 2 my (not fully-thought-through) intuition about how to make the values consistent is be:

• module_definition*: Wherever module_definition is set it should be replaced by setting a channel identifier. E.g. one could add a keyword parameter channel=None to update_state if the device has only one device and a channel=<identifier> where multiple channels are present as well as updating the channel of concern whenever originally module_definition/hash was used/updated before. For full consistency the Duofern.set would also need to become channel-aware in the same manner (only a few lines dealing with device ids of lenght 8 would be affected).
• code: would be used in the rademacher sense only, together with a channel identifier where a channel is present
• hash: This is almost unused in pyduofern -- to be frank I ported the file top-to-bottom from perl and translated code for "hash" without knowing what exactly it would be used for later on. I think it can be gotten rid of and if actively used should be replaceable by calls to update_state or delete_state.
• device_id: could be replaced by code together with a channel identifier.

One issue I have with the explicit channel: it would add dimensions to the modules['by_code'] dict -- I find it simpler to leave that as-is and use the FHEM notion of 2(3) devices, one for each channel. (your suggestion 1)

Finally: I wasn't aware that there are Rademacher manuals out that mention the code -- the one coming with my roller shutters is not explaining what happens in the background and to be frank I was already wondering how the knowledge required for the FHEM implementation was gathered. Do you have a link to a pdf version?

[koalo]

The only manual I have is this one https://service.rademacher.de/hc/article_attachments/360007625334/VBD_622-1-(06.18)-DF-Universal-Aktor-9470-2-DE_Internet.pdf but even there a "Funkcode" is repeatedly mentioned. E.g. in 9.1. There is AFAIK no communication documentation and the FHEM module was built with a lot of reverse engineering skills.

I also initially preferred option 2, but see that option 1 is probably faster to implement. However, maybe we should not fully give up on this yet. E.g. the dict maybe does not even need another dimension. Maybe it is sufficient to just add another parameter channel_count and drop the chanNo since there is currently not much information in it apart from the code itself (and the name that does not seem to be really used) and if we always keep the channel number as a separate variable together with the code at all places where it is relevant it could be sufficient.

[gluap]

@koalo I had a shot at option 2 in https://github.com/gluap/pyduofern/tree/feature/dualactor, didn't want to commit here before you can have a look. What do you think? Would this be usable for you?

Caveat: I din't touch the homeassistant module yet.
My only step so far: I am setting the channel number whenever originally "hash" was updated and I am (for the time being) postfixing the status variables for the non-default channels with the channel number. To make it usable in homeassistant changes to the light module will be required -- because no extra module IDs are created any more, homeassistant will not know about the non-default channels.

Was your patch already working on your end? If so it seems we can disregard chanNo.
It seems that currently chanNo is hard-wired to "01" because the if where it might get a different assignment is always False. As far as I can find using text search no codepath ever adds the key chanNo to the dict.

if 'chanNo' in self.modules['by_code'][code]: # always false?!
    chanNo = self.modules['by_code'][code]['chanNo']

But probably that was a bug I introduced while porting -- if so we could add "channel" as a keyword parameter also to Duofern.set() and use it to set the channel number.

[koalo]

@koalo I had a shot at option 2 in https://github.com/gluap/pyduofern/tree/feature/dualactor, didn't want to commit here before you can have a look. What do you think? Would this be usable for you?

I this is a very good way to go forward. I made some comments in the respective commit.

Caveat: I din't touch the homeassistant module yet.
My only step so far: I am setting the channel number whenever originally "hash" was updated and I am (for the time being) postfixing the status variables for the non-default channels with the channel number. To make it usable in homeassistant changes to the light module will be required -- because no extra module IDs are created any more, homeassistant will not know about the non-default channels.

I think that is a completely valid approach. It would be good if you could handle the comments I made in your commit then I can happily try to adapt the homeassistant module and test it with my setup at home.

Was your patch already working on your end? If so it seems we can disregard chanNo.
It seems that currently chanNo is hard-wired to "01" because the if where it might get a different assignment is always False. As far as I can find using text search no codepath ever adds the key chanNo to the dict.

if 'chanNo' in self.modules['by_code'][code]: # always false?!
    chanNo = self.modules['by_code'][code]['chanNo']

But probably that was a bug I introduced while porting -- if so we could add "channel" as a keyword parameter also to Duofern.set() and use it to set the channel number.

These lines were responsible for setting the chanNo, so it was actually used and required:

        if len(code) == 8:
          self.modules['by_code'][code]['chanNo'] = code[6:]

You removed them with 9c9be2a, but as I wrote in one of the comments, using the channel parameter instead of the chanNo would be the preferred way. It is, however, required to pad the channel number with a zero before applying buf = buf.replace("kk", chanNo).

[gluap]

Thank you for your thorough review. I added the changes you proposed, I left some of the removed lines commented out and will remove them once it is tested a bit more thoroughly.

edit: I added a set listing the available channels in Duofern().modules['by_code'][code]['channels'] so it is transparent to the outside which channels exist for each device.

[coveralls]

Pull Request Test Coverage Report for Build 55

• 86 of 110 (78.18%) changed or added relevant lines in 4 files are covered.
• 2 unchanged lines in 1 file lost coverage.
• Overall coverage increased (+0.1%) to 75.947%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
pyduofern/tests/test_replay.py	7	8	87.5%
pyduofern/duofern_stick.py	3	9	33.33%
pyduofern/duofern.py	75	92	81.52%

Files with Coverage Reduction	New Missed Lines	%
pyduofern/duofern.py	2	63.05%

Totals
Change from base Build 33:	0.1%
Covered Lines:	1484
Relevant Lines:	1954

---

💛 - Coveralls

[koalo]

Looks very good, thanks! I made some minor modifications and updated the custom component. It now works again with my setup. I have, however, no covers, so it would be good if you test it again on your side.

[gluap]

@koalo it seems my cover devices do work with the changes. Thank you for the PR as well as the updates to the custom components as well as the fixes on the way. Functionally I think we can commit this. One more request though to ensure the feature stays stable for the future:

Could you set

recording = True
recording_dir = <some directory of your choice>

in your/a duofern.json config file to produce replay files.

Once this is set could you run on and off for a dual channel actor once for each channel for instance using the (admittedly poorly designed) CLI.

Finally could you send me / pastebin / gist the replay files after replacing your 4 digit system code inside the files with ffff (look at the files in tests/replaydata as an example). The replay mechanism is not perfect yet so the replay likely needs some twiddling (which I would do) before replaying successfully.

The purpose is to cover the dual channel actor in the replay unit tests.

If you are courageous you could also try to directly drop it in replaydata and run the unit tests, but that will likely break the unit tests without fiddling with the replay beforehand unless I misremember the amount of work required for the existing replays.

[koalo]

I have performed the recording and pushed it to the branch, but as you expected, the test fails now.