Handle catch-up data

[limes007]

Today my inverter was offline for many hours (9:00-18:00) and I noticed some interesting behavior:

When the connection is re-established, the inverter not only sends its current data every 5 minutes. It has been sending 10 additional packets every 5 minutes for ~1 hour. These additional data packets appear to contain data from the time the inverter was offline.

This 'catch-up' data is then used at talent-monitoring.com to fill the gap in the graph for the time the inverter was offline.

Screenshot of 18:30, where data up to 12:30 was subsequently filled.

Screenshot of 19:50, data was subsequently filled completely.

The proxy doesn't distinguish between normal data packets and these catch-up data packets. So all packages are handled as current data, so the chart in Home Assistant looks like this.

I don't know if it will be possible to process this 'catch-up' data with the correct time stamp and then fill in the data in the Home Assistant. At least it would be good to ignore this data if it cannot be handled appropriately.

[s-allius]

The GEN3PLUS inverter sends the 'catch-up' data with every connection. It would be very easy to send this data to the MQTT broker with the old timestamp.
But Home-Assistant only accept data in realtime. My knowledge is, that there is no solution to add the data to the home-assistant database by MQTT. You need an integration running on Home Assistant to write the data into the database.
Maybe that is possible with other MQTT clients.
Do you have a GEN3PLUS (e.g. MS2000) inverter or one of the GEN3 (e.g MS600) series?

[limes007]

I understand, I already assumed, that this will be not easy possible.

I have an older model TSOL-MS600, but with probably exotic firmware RSW_400_V1.00.20

[s-allius]

Can you send me a trace or a dump of one of this additional packets with catchup data?
Maybe I can add a filter or a timestamp to the MQTT packets

[limes007]

Of course. I assume, there are sensible private data inside these traces, which should not published. How can I remove them?

[s-allius]

Pls send a mail: [email protected]

[s-allius]

Thank you for the trace.
This behavior only seems to be implemented in the rare version SW_400_V1.00.20. However, the GEN3PLUS inverters also do this.
In a later version I will include the time stamps in the MQTT messages and install filters in HA via autoconfiguration. Then the data will not be transferred to HA, but anyone using a different solution will be able to evaluate it. Perhaps there is also a solution for retroactively saving the data in the HA database

[limes007]

Great, thank you.

[limes007]

@s-allius I've been using version 0.8.0 for a few days now, and at the moment I seem to be having a lot of connection problems, so a lot of data is being sent as a catch-up block.
Do I need to do some configuration in HA to enable the mentioned filter for the catch-up data? Currently, the chart in HA looks like this.

[s-allius]

Please try the release candidate v0.8.1-rc.1. There is a hot fix which should fix the connection problems for the GEN3 inverters with version 0.8.0.

The filter for the catch-up data in 0.8.0 is only available for GEN3PLUS inverters. For the GEN3 ones I will bring this with version 0.9

[limes007]

Ok, thank you. I'm on v0.8.1-rc.1 now.