KeyError:'dataList' #28

Krishnaj-20 · 2022-04-14T11:12:25Z

Hi

This might be a stupid question but when i try and run the python script I get KeyError:'dataList'

Would you be able to assist with this?

Regards

srk23 · 2022-05-11T12:49:22Z

I am getting the same error...

When I run manually using python, I get:

Traceback (most recent call last):
  File "run.py", line 14, in <module>
    main()
  File "run.py", line 10, in main
    solarman.main()
  File "/opt/solarman-mqtt/solarman/__init__.py", line 184, in main
    single_run(args.file)
  File "/opt/solarman-mqtt/solarman/__init__.py", line 62, in single_run
    inverter_data_list = ConstructData(inverter_data).device_current_data
  File "/opt/solarman-mqtt/solarman/api.py", line 97, in __init__
    self.device_current_data = self.construct_data()
  File "/opt/solarman-mqtt/solarman/api.py", line 104, in construct_data
    if self.data["dataList"]:
KeyError: 'dataList'

srk23 · 2022-05-11T13:36:17Z

Duh. Just realised that the inverter and logger IDs should be the serial number and not the ID - working now!

mpepping · 2022-05-13T20:27:25Z

Good to hear, @srk23. @Krishnaj-20, is that also applicable to your setup? I'm in the process of adding better error handling for this.

giovanne123 · 2022-07-19T12:20:47Z

Hm, same error.
It was working and suddenly stopped working.
No idea :-(

INFO:root:2022-07-19 14:03:27 - Inverter DeviceState: 1 -> Publishing to MQTT ...
INFO:root:2022-07-19 14:04:29 - Inverter DeviceState: 1 -> Publishing to MQTT ...
INFO:root:2022-07-19 14:05:32 - Inverter DeviceState: 1 -> Publishing to MQTT ...
INFO:root:2022-07-19 14:06:34 - Inverter DeviceState: 1 -> Publishing to MQTT ...
INFO:root:2022-07-19 14:07:36 - Inverter DeviceState: 1 -> Publishing to MQTT ...
INFO:root:2022-07-19 14:08:38 - Inverter DeviceState: 1 -> Publishing to MQTT ...
Traceback (most recent call last):
  File "run.py", line 14, in <module>
    main()
  File "run.py", line 10, in main
    solarman.main()
  File "/home/pi/solarman-mqtt/solarman/__init__.py", line 184, in main
    single_run(args.file)
  File "/home/pi/solarman-mqtt/solarman/__init__.py", line 62, in single_run
    inverter_data_list = ConstructData(inverter_data).device_current_data
  File "/home/pi/solarman-mqtt/solarman/api.py", line 97, in __init__
    self.device_current_data = self.construct_data()
  File "/home/pi/solarman-mqtt/solarman/api.py", line 104, in construct_data
    if self.data["dataList"]:
KeyError: 'dataList'

giovanne123 · 2022-07-19T12:53:58Z

Hm, now it is starting to work again:

KeyError: 'dataList'
Traceback (most recent call last):
  File "run.py", line 14, in <module>
    main()
  File "run.py", line 10, in main
    solarman.main()
  File "/home/pi/solarman-mqtt/solarman/__init__.py", line 184, in main
    single_run(args.file)
  File "/home/pi/solarman-mqtt/solarman/__init__.py", line 62, in single_run
    inverter_data_list = ConstructData(inverter_data).device_current_data
  File "/home/pi/solarman-mqtt/solarman/api.py", line 97, in __init__
    self.device_current_data = self.construct_data()
  File "/home/pi/solarman-mqtt/solarman/api.py", line 104, in construct_data
    if self.data["dataList"]:
KeyError: 'dataList'
^C
root@HRG-CoreServer:/home/pi/solarman-mqtt# while true; do python3 run.py -s; sleep 60s; done
INFO:root:2022-07-19 14:51:24 - Inverter DeviceState: 1 -> Publishing to MQTT ...
INFO:root:2022-07-19 14:52:26 - Inverter DeviceState: 1 -> Publishing to MQTT ...
INFO:root:2022-07-19 14:53:29 - Inverter DeviceState: 1 -> Publishing to MQTT ...

was working until now without problem (without changes), now the error is back:

INFO:root:2022-07-20 10:47:41 - Inverter DeviceState: 1 -> Publishing to MQTT ...
INFO:root:2022-07-20 10:48:44 - Inverter DeviceState: 1 -> Publishing to MQTT ...
Traceback (most recent call last):
  File "run.py", line 14, in <module>
    main()
  File "run.py", line 10, in main
    solarman.main()
  File "/home/pi/solarman-mqtt/solarman/__init__.py", line 184, in main
    single_run(args.file)
  File "/home/pi/solarman-mqtt/solarman/__init__.py", line 62, in single_run
    inverter_data_list = ConstructData(inverter_data).device_current_data
  File "/home/pi/solarman-mqtt/solarman/api.py", line 97, in __init__
    self.device_current_data = self.construct_data()
  File "/home/pi/solarman-mqtt/solarman/api.py", line 104, in construct_data
    if self.data["dataList"]:
KeyError: 'dataList'

will wait until it starts working again ;-)

Continuing to work :-)

Traceback (most recent call last):
  File "run.py", line 14, in <module>
    main()
  File "run.py", line 10, in main
    solarman.main()
  File "/home/pi/solarman-mqtt/solarman/__init__.py", line 184, in main
    single_run(args.file)
  File "/home/pi/solarman-mqtt/solarman/__init__.py", line 62, in single_run
    inverter_data_list = ConstructData(inverter_data).device_current_data
  File "/home/pi/solarman-mqtt/solarman/api.py", line 97, in __init__
    self.device_current_data = self.construct_data()
  File "/home/pi/solarman-mqtt/solarman/api.py", line 104, in construct_data
    if self.data["dataList"]:
KeyError: 'dataList'
INFO:root:2022-07-20 11:25:01 - Inverter DeviceState: 1 -> Publishing to MQTT ...
INFO:root:2022-07-20 11:26:03 - Inverter DeviceState: 1 -> Publishing to MQTT ...

Some idea to debug the reason?

finfin82 · 2022-10-08T16:29:02Z

Same here - are there any news for that problem?
I realy wanna use this cool solution to grab my inverter data.
Will there be a fix in the main-branch for that?
including property-naming (loggerid-> loggerSN)

In some occasions, the `dataList` keys is missing from the API data.

mpepping · 2022-10-09T11:29:40Z

Seem like that dataList key is missing in some occurances? There's a small update in the main-branch / v0.6.2 that should address this. Can you give this a go, @giovanne123, @finfin82 ?

docker pull ghcr.io/mpepping/solarman-mqtt:0.6.2 or docker pull ghcr.io/mpepping/solarman-mqtt:latest

giovanne123 · 2022-10-12T15:00:05Z

Thanks, I will give it a try.
Yesterday I got all the time:

PiServer:/home/pi/solarman-mqtt/NEW_10-2022/solarman-mqtt#  python3 run.py -s
Traceback (most recent call last):
  File "run.py", line 14, in <module>
    main()
  File "run.py", line 10, in main
    solarman.main()
  File "/home/pi/solarman-mqtt/NEW_10-2022/solarman-mqtt/solarman/__init__.py", line 184, in main
    single_run(args.file)
  File "/home/pi/solarman-mqtt/NEW_10-2022/solarman-mqtt/solarman/__init__.py", line 76, in single_run
    inverter_device_state = inverter_data["deviceState"]
KeyError: 'deviceState'

I'm not sure why.
But today it started to work:

PiServer:/home/pi/solarman-mqtt/NEW_10-2022/solarman-mqtt# python3 run.py -s
2022-10-12 16:53:14 - Inverter DeviceState: 1 -> Publishing to MQTT ...
2022-10-12 16:54:16 - Inverter DeviceState: 1 -> Publishing to MQTT ...
2022-10-12 16:55:19 - Inverter DeviceState: 1 -> Publishing to MQTT ...
2022-10-12 16:56:21 - Inverter DeviceState: 1 -> Publishing to MQTT ...

I will check if it will run without dataList error...

Update: After ~4 minutes it stops working
Now I get KeyError: 'deviceState' instead :-(

...
2022-10-12 17:00:51 - Inverter DeviceState: 1 -> Publishing to MQTT ...
2022-10-12 17:01:53 - Inverter DeviceState: 1 -> Publishing to MQTT ...
Traceback (most recent call last):
  File "run.py", line 14, in <module>
    main()
  File "run.py", line 10, in main
    solarman.main()
  File "/home/pi/solarman-mqtt/NEW_10-2022/solarman-mqtt/solarman/__init__.py", line 184, in main
    single_run(args.file)
  File "/home/pi/solarman-mqtt/NEW_10-2022/solarman-mqtt/solarman/__init__.py", line 76, in single_run
    inverter_device_state = inverter_data["deviceState"]
KeyError: 'deviceState'
Traceback (most recent call last):
  File "run.py", line 14, in <module>
    main()
  File "run.py", line 10, in main
    solarman.main()
  File "/home/pi/solarman-mqtt/NEW_10-2022/solarman-mqtt/solarman/__init__.py", line 184, in main
    single_run(args.file)
  File "/home/pi/solarman-mqtt/NEW_10-2022/solarman-mqtt/solarman/__init__.py", line 76, in single_run
    inverter_device_state = inverter_data["deviceState"]
KeyError: 'deviceState'

After a while it was continuing to work...


Traceback (most recent call last):
  File "run.py", line 14, in <module>
    main()
  File "run.py", line 10, in main
    solarman.main()
  File "/home/pi/solarman-mqtt/NEW_10-2022/solarman-mqtt/solarman/__init__.py", line 184, in main
    single_run(args.file)
  File "/home/pi/solarman-mqtt/NEW_10-2022/solarman-mqtt/solarman/__init__.py", line 76, in single_run
    inverter_device_state = inverter_data["deviceState"]
KeyError: 'deviceState'
Traceback (most recent call last):
  File "run.py", line 14, in <module>
    main()
  File "run.py", line 10, in main
    solarman.main()
  File "/home/pi/solarman-mqtt/NEW_10-2022/solarman-mqtt/solarman/__init__.py", line 184, in main
    single_run(args.file)
  File "/home/pi/solarman-mqtt/NEW_10-2022/solarman-mqtt/solarman/__init__.py", line 76, in single_run
    inverter_device_state = inverter_data["deviceState"]
KeyError: 'deviceState'
2022-10-12 17:15:22 - Inverter DeviceState: 1 -> Publishing to MQTT ...
2022-10-12 17:16:24 - Inverter DeviceState: 1 -> Publishing to MQTT ...
2022-10-12 17:17:26 - Inverter DeviceState: 1 -> Publishing to MQTT ...
2022-10-12 17:18:29 - Inverter DeviceState: 1 -> Publishing to MQTT ...
2022-10-12 17:19:31 - Inverter DeviceState: 1 -> Publishing to MQTT ...
2022-10-12 17:20:33 - Inverter DeviceState: 1 -> Publishing to MQTT ...

mpepping · 2022-10-15T10:06:42Z

Have seen API rate limiting in the past for myself. The (default) interval of 5 minutes works OK for me. A polling interval of a minute (your example) doesn't seem that frequent .. but maybe it's useful to test with a higher interval.

giovanne123 · 2022-10-28T08:43:50Z

I will try to change to 5min,
but otherside I have running it again since my last post for now ~2week without a problem with 1min intervall. It's wirred and a not reproduceable behaviour:-(

giovanne123 · 2022-12-15T17:12:57Z

FYI, no change for me with 5min intervall too:

With newest I get - "deviceState" error:

Traceback (most recent call last):
  File "run.py", line 14, in <module>
    main()
  File "run.py", line 10, in main
    solarman.main()
  File "/home/pi/solarman-mqtt/NEW_10-2022/solarman-mqtt/solarman/__init__.py", line 184, in main
    single_run(args.file)
  File "/home/pi/solarman-mqtt/NEW_10-2022/solarman-mqtt/solarman/__init__.py", line 76, in single_run
    inverter_device_state = inverter_data["deviceState"]
KeyError: 'deviceState'

With older version - "dataList" error:

Traceback (most recent call last):
  File "run.py", line 14, in <module>
    main()
  File "run.py", line 10, in main
    solarman.main()
  File "/home/pi/solarman-mqtt/solarman/__init__.py", line 184, in main
    single_run(args.file)
  File "/home/pi/solarman-mqtt/solarman/__init__.py", line 62, in single_run
    inverter_data_list = ConstructData(inverter_data).device_current_data
  File "/home/pi/solarman-mqtt/solarman/api.py", line 97, in __init__
    self.device_current_data = self.construct_data()
  File "/home/pi/solarman-mqtt/solarman/api.py", line 104, in construct_data
    if self.data["dataList"]:
KeyError: 'dataList'

I think we can close the issue, because only I have this problem and it seems to work for the others.
(maybe I will also change to modbus to read the inverter data locally without cn cloud)

mpepping · 2023-01-04T13:21:08Z

Interesting error, @giovanne123. I'm curious to see what data is in the debug output. Is it's possible for you to do a run with "debug": true added to your config file? I'm especially interested in the block:

INFO:root:{
    "code": null,
    "deviceId": 123456790,
    "deviceSn": "123456790123456790",
    "deviceState": 3,
    "deviceType": "INVERTER",
    "msg": null,
    "requestId": "123456790",
    "success": true
}

Just to learn what is in the deviceState key or the whole inverter_date object in the data you receive back, on the moment the run errors.

Now, a fix is added in the latest code in the main branch, to catch the KeyError: 'deviceState' error .. but it will effectively just skip and try again in the next iteration. Since you said that the error occurs every now and then, this should handle that. Still it would be nice to know what is in the actual data you get back from the API.

giovanne123 · 2023-01-04T19:02:03Z

Hm, there is no (or empty or not found) INVERTER part that time the error occurs:
This is the output (I inserted some xxx for some data)

python3 run.py -s
INFO:root:{
    "batteryPower": null,
    "batterySoc": null,
    "chargePower": null,
    "code": null,
    "dischargePower": null,
    "generationPower": 2.0,
    "generationTotal": 253.0,
    "gridPower": null,
    "irradiateIntensity": null,
    "lastUpdateTime": 1672845175.0,
    "msg": null,
    "purchasePower": null,
    "requestId": "xxxxxxxxxxxxxxxxxxx",
    "success": true,
    "usePower": null,
    "wirePower": null
}
INFO:root:{
    "code": "2101xxx",
    "msg": "device not found",
    "requestId": "xxxxxxxxxxxxxxxxxxx",
    "success": false
}
INFO:root:{}
INFO:root:{
    "code": null,
    "deviceId": 214xxxxxx,
    "deviceSn": "4111xxxxxx",
    "deviceState": 3,
    "deviceType": "COLLECTOR",
    "msg": null,
    "requestId": "xxxxxxxxxxxxxxxxxxx",
    "success": true
}
INFO:root:{
    "AT+UPGRADE_Command_Supported": "255",
    "Batch_Command_Supported": "1",
    "Data_Acquisition_Period": "60",
    "Data_Uploading_Period": "1",
    "Embedded_Device_SN": "4111xxxxxx",
    "Extended_Function": "7",
    "Extended_System_Version": "V1.1.00.0F",
    "Heart_Rate": "120",
    "IV_Curve_Supported": "1",
    "Max._No._of_Connected_Devices": "1",
    "Method_Of_Protocol_Upgrade": "255",
    "Module_Version_No": "MW3_15U_5406_1.47",
    "Offset_time": "1672844923",
    "Signal_Strength": "49",
    "Support_AntiBackflow_Function_Instructions": "Enable",
    "Support_Reporting_Upgrading_Progress": "1",
    "Total_running_time": "1"
}
Traceback (most recent call last):
  File "run.py", line 14, in <module>
    main()
  File "run.py", line 10, in main
    solarman.main()
  File "/home/pi/solarman-mqtt/NEW_10-2022/solarman-mqtt/solarman/__init__.py", line 184, in main
    single_run(args.file)
  File "/home/pi/solarman-mqtt/NEW_10-2022/solarman-mqtt/solarman/__init__.py", line 76, in single_run
    inverter_device_state = inverter_data["deviceState"]
KeyError: 'deviceState'

and this is the "device not found" and "empty" section if the error doesn't occur:
(I have a "MICRO_INVERTER" - Bosswerk MI600)

INFO:root:{
    "code": null,
    "deviceId": 214xxxxxx,
    "deviceSn": "220xxxxxx",
    "deviceState": 1,
    "deviceType": "MICRO_INVERTER",
    "msg": null,
    "requestId": "5b280xxxxxxbc0",
    "success": true
}
INFO:root:{
    "AC_Current_1": "0.00",
    "AC_Output_Frequency_1": "49.89",
    "AC_Radiator_Temp": "13.30",
    "AC_Voltage_1": "240.00",
    "AC_version_number": "V0.1.0.0",
    "Communication_Protocol_Version": "V0.2.0.1",
    "DC_Current_PV1": "0.10",
    "DC_Current_PV2": "0.10",
    "DC_Current_PV3": "0.00",
    "DC_Current_PV4": "0.00",
    "DC_Master_Firmware_Version": "V0.0.0.4",
    "DC_Power_PV1": "2.80",
    "DC_Power_PV2": "2.15",
    "DC_Power_PV3": "0.00",
    "DC_Power_PV4": "0.00",
    "DC_Slave_Firmware_Version": "V0.2.0.1",
    "DC_Voltage_PV1": "28.00",
    "DC_Voltage_PV2": "21.50",
    "DC_Voltage_PV3": "0.00",
    "DC_Voltage_PV4": "0.00",
    "Daily_Production(Active)": "0.10",
    "Daily_Production_1": "0.00",
    "Daily_Production_2": "0.00",
    "Daily_Production_3": "0.00",
    "Daily_Production_4": "0.00",
    "Day": "5",
    "Debug_Info_1": "0",
    "Debug_Info_2": "0",
    "Device_Type": "4",
    "GFDI\u00a0Enabled": "Disable",
    "Grid-connected\u00a0Standard": "0",
    "Grid\u00a0Frequency_Lower_Limit": "47.50",
    "Grid\u00a0Frequency_Upper_Limit": "52.00",
    "Grid\u00a0Voltage_Lower_Limit": "180.00",
    "Grid\u00a0Voltage_Upper_Limit": "275.00",
    "Hardware_Version": "V0.3.2.8",
    "Hour": "15",
    "Islanding\u00a0Protection\u00a0Enabled": "Enable",
    "MECD_Status": "1",
    "Micro_Inverte_System_Flag_Bits": "1",
    "Micro_Inverter_Port_1": "2204xxxxxx0-1",
    "Micro_Inverter_Port_2": "2204xxxxxx0-2",
    "Micro_Inverter_Port_3": "2204xxxxxx0-3",
    "Micro_Inverter_Port_4": "2204xxxxxx0-4",
    "Minute": "40",
    "Month": "1",
    "Number_Of_MPPT_Paths": "2",
    "Number_Of_Phases": "1",
    "Overfrequency_And_Load_Reduction_Percentage": "44",
    "Overfrequency_And_Load_Reduction_Starting_Point": "50.20",
    "Power_Rating": "0",
    "Rated_Power": "600.00",
    "Running_Status": "4",
    "SN": "2204134270",
    "Seconds": "2",
    "Start-up\u00a0Self-checking\u00a0Time": "60",
    "Total_AC_Output_Power(Active)": "7",
    "Total_Production(Active)": "253.20",
    "Total_Production_1": "125.00",
    "Total_Production_2": "118.00",
    "Total_Production_3": "0.00",
    "Total_Production_4": "0.00",
    "Year": "23"
}

mpepping · 2023-01-05T20:08:28Z

So the object is completely empty. Wasn’t expecting that! Question is then if the data is also not present in the Solarman cloud graphs. That would suggest that it’s also missing from the data that was sent to the cloud service.
Version v0.8.0 (or the main branch) will handle this error, by skipping the run and just wait for the next iteration. Can’t think of a valid way to deal with this in an other way for now.

giovanne123 · 2023-01-05T20:24:29Z

OK, I will update to v0.8.0
And we can close this issue, because nobody else seems to have the same problem.
I will only use this on demand to read e.g. firmware/settings infos before/after requesting an update...
For live data I already switched to local modbus readings.

mpepping added a commit that referenced this issue Oct 9, 2022

Pass is dataList key is missing (#28)

dd21fdf

In some occasions, the `dataList` keys is missing from the API data.

mpepping added a commit that referenced this issue Oct 9, 2022

Pass is dataList key is missing (#28) (#38)

d1a16a5

In some occasions, the `dataList` keys is missing from the API data.

mpepping self-assigned this Oct 9, 2022

mpepping closed this as completed Jan 5, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

KeyError:'dataList' #28

KeyError:'dataList' #28

Krishnaj-20 commented Apr 14, 2022

srk23 commented May 11, 2022 •

edited

Loading

srk23 commented May 11, 2022

mpepping commented May 13, 2022

giovanne123 commented Jul 19, 2022

giovanne123 commented Jul 19, 2022 •

edited

Loading

finfin82 commented Oct 8, 2022

mpepping commented Oct 9, 2022

giovanne123 commented Oct 12, 2022 •

edited

Loading

mpepping commented Oct 15, 2022

giovanne123 commented Oct 28, 2022 •

edited

Loading

giovanne123 commented Dec 15, 2022

mpepping commented Jan 4, 2023

giovanne123 commented Jan 4, 2023 •

edited

Loading

mpepping commented Jan 5, 2023

giovanne123 commented Jan 5, 2023

KeyError:'dataList' #28

KeyError:'dataList' #28

Comments

Krishnaj-20 commented Apr 14, 2022

srk23 commented May 11, 2022 • edited Loading

srk23 commented May 11, 2022

mpepping commented May 13, 2022

giovanne123 commented Jul 19, 2022

giovanne123 commented Jul 19, 2022 • edited Loading

finfin82 commented Oct 8, 2022

mpepping commented Oct 9, 2022

giovanne123 commented Oct 12, 2022 • edited Loading

mpepping commented Oct 15, 2022

giovanne123 commented Oct 28, 2022 • edited Loading

giovanne123 commented Dec 15, 2022

mpepping commented Jan 4, 2023

giovanne123 commented Jan 4, 2023 • edited Loading

mpepping commented Jan 5, 2023

giovanne123 commented Jan 5, 2023

srk23 commented May 11, 2022 •

edited

Loading

giovanne123 commented Jul 19, 2022 •

edited

Loading

giovanne123 commented Oct 12, 2022 •

edited

Loading

giovanne123 commented Oct 28, 2022 •

edited

Loading

giovanne123 commented Jan 4, 2023 •

edited

Loading