Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ICD] Add ICDM 5.1 Automated Test Script #34817

Merged
merged 21 commits into from
Sep 6, 2024

Conversation

mkardous-silabs
Copy link
Contributor

@mkardous-silabs mkardous-silabs commented Aug 6, 2024

Description

Adds the ICDM 5.1 Test Case automated script.
Test Plan PR: https://github.com/CHIP-Specifications/chip-test-plans/pull/4432

Tests

CI

Copy link

semanticdiff-com bot commented Aug 6, 2024

Review changes with SemanticDiff.

Analyzed 2 of 4 files.

Overall, the semantic diff is 28% smaller than the GitHub diff.

Filename Status
src/python_testing/TC_ICDM_5_1.py Unsupported file format
✔️ src/python_testing/mdns_discovery/mdns_discovery.py 28.2% smaller
✔️ src/app/tests/suites/manualTests.json Analyzed
src/app/tests/suites/certification/Test_TC_ICDM_5_1.yaml Unsupported file format

@github-actions github-actions bot added app github workflows tests matter-1.4-te2-script-change Script changes before end of Matter 1.4 TE2 labels Aug 6, 2024
@mergify mergify bot removed the conflict label Aug 20, 2024
Copy link

github-actions bot commented Aug 20, 2024

PR #34817: Size comparison from 068da61 to efaefec

Full report (77 builds for bl602, bl702, bl702l, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
platform target config section 068da61 efaefec change % change
bl602 lighting-app bl602 FLASH 1278282 1278282 0 0.0
RAM 95880 95880 0 0.0
bl602+mfd FLASH 1292392 1292392 0 0.0
RAM 96024 96024 0 0.0
bl602+rpc FLASH 1317246 1317246 0 0.0
RAM 104304 104304 0 0.0
bl702 lighting-app bl702 FLASH 944186 944186 0 0.0
RAM 15209 15209 0 0.0
bl702+mfd FLASH 946586 946586 0 0.0
RAM 15353 15353 0 0.0
bl702+rpc FLASH 1039242 1039242 0 0.0
RAM 24221 24221 0 0.0
bl706-eth FLASH 646848 646848 0 0.0
RAM 25305 25305 0 0.0
bl706-wifi FLASH 894448 894448 0 0.0
RAM 14525 14525 0 0.0
bl702l lighting-app bl702l FLASH 960590 960590 0 0.0
RAM 16868 16868 0 0.0
bl702l+mfd FLASH 963468 963468 0 0.0
RAM 17012 17012 0 0.0
cyw30739 light CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 672465 672465 0 0.0
RAM 78300 78300 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 692325 692325 0 0.0
RAM 80932 80932 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 692325 692325 0 0.0
RAM 80932 80932 0 0.0
CYW930739M2EVB-02 unknown 2040 2040 0 0.0
FLASH 649261 649261 0 0.0
RAM 73368 73368 0 0.0
light-switch CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 610929 610929 0 0.0
RAM 71292 71292 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 630557 630557 0 0.0
RAM 73844 73844 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 630557 630557 0 0.0
RAM 73844 73844 0 0.0
lock CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 628969 628969 0 0.0
RAM 74308 74308 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 648677 648677 0 0.0
RAM 76860 76860 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 648677 648677 0 0.0
RAM 76860 76860 0 0.0
thermostat CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 601917 601917 0 0.0
RAM 68340 68340 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 621777 621777 0 0.0
RAM 70972 70972 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 621777 621777 0 0.0
RAM 70972 70972 0 0.0
efr32 lighting-app BRD4187C FLASH 930564 930564 0 0.0
RAM 135096 135096 0 0.0
lock-app BRD4338a FLASH 740564 740556 -8 -0.0
RAM 208488 208488 0 0.0
window-app BRD4187C FLASH 1015656 1015656 0 0.0
RAM 127036 127036 0 0.0
esp32 all-clusters-app c3devkit DRAM 94192 94192 0 0.0
FLASH 1532632 1532632 0 0.0
IRAM 82538 82538 0 0.0
m5stack DRAM 115128 115128 0 0.0
FLASH 1542398 1542398 0 0.0
IRAM 117039 117039 0 0.0
linux air-purifier-app debug unknown 4624 4624 0 0.0
FLASH 2755419 2755419 0 0.0
RAM 128408 128408 0 0.0
all-clusters-app debug unknown 5408 5408 0 0.0
FLASH 6037850 6037850 0 0.0
RAM 509736 509736 0 0.0
all-clusters-minimal-app debug unknown 5304 5304 0 0.0
FLASH 5377354 5377354 0 0.0
RAM 240104 240104 0 0.0
bridge-app debug unknown 5288 5288 0 0.0
FLASH 4729216 4729216 0 0.0
RAM 216952 216952 0 0.0
chip-tool debug unknown 5832 5832 0 0.0
FLASH 12488672 12488672 0 0.0
RAM 557202 557202 0 0.0
chip-tool-ipv6only arm64 unknown 20312 20312 0 0.0
FLASH 11164300 11164300 0 0.0
RAM 607272 607272 0 0.0
fabric-admin debug unknown 5664 5664 0 0.0
FLASH 10859615 10859615 0 0.0
RAM 555402 555402 0 0.0
fabric-bridge-app debug unknown 4520 4520 0 0.0
FLASH 4550056 4550056 0 0.0
RAM 203648 203648 0 0.0
lighting-app debug+rpc+ui unknown 5936 5936 0 0.0
FLASH 5663777 5663777 0 0.0
RAM 228008 228008 0 0.0
lock-app debug unknown 5224 5224 0 0.0
FLASH 4784562 4784562 0 0.0
RAM 203736 203736 0 0.0
ota-provider-app debug unknown 4600 4600 0 0.0
FLASH 4424036 4424036 0 0.0
RAM 197720 197720 0 0.0
ota-requestor-app debug unknown 4536 4536 0 0.0
FLASH 4562654 4562654 0 0.0
RAM 202256 202256 0 0.0
shell debug unknown 4176 4176 0 0.0
FLASH 3075197 3075197 0 0.0
RAM 158936 158936 0 0.0
thermostat-no-ble arm64 unknown 9352 9352 0 0.0
FLASH 4364340 4364340 0 0.0
RAM 242416 242416 0 0.0
tv-app debug unknown 5496 5496 0 0.0
FLASH 6007061 6007061 0 0.0
RAM 582968 582968 0 0.0
tv-casting-app debug unknown 5168 5168 0 0.0
FLASH 10692765 10692765 0 0.0
RAM 644280 644280 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 FLASH 909344 909344 0 0.0
RAM 142211 142211 0 0.0
nrf7002dk_nrf5340_cpuapp FLASH 881120 881120 0 0.0
RAM 140350 140350 0 0.0
all-clusters-minimal-app nrf52840dk_nrf52840 FLASH 841176 841176 0 0.0
RAM 141030 141030 0 0.0
nxp contact k32w0+release FLASH 577580 577580 0 0.0
RAM 70444 70444 0 0.0
k32w1+release FLASH 590208 590208 0 0.0
RAM 62488 62488 0 0.0
light k32w0+release FLASH 613008 613008 0 0.0
RAM 69916 69916 0 0.0
k32w1+release FLASH 674880 674880 0 0.0
RAM 71472 71472 0 0.0
psoc6 all-clusters cy8ckit_062s2_43012 FLASH 1639508 1639508 0 0.0
RAM 210904 210904 0 0.0
all-clusters-minimal cy8ckit_062s2_43012 FLASH 1544260 1544260 0 0.0
RAM 207624 207624 0 0.0
light cy8ckit_062s2_43012 FLASH 1464028 1464028 0 0.0
RAM 200728 200728 0 0.0
lock cy8ckit_062s2_43012 FLASH 1463236 1463236 0 0.0
RAM 225064 225064 0 0.0
qpg lighting-app qpg6105+debug FLASH 656116 656116 0 0.0
RAM 105128 105128 0 0.0
lock-app qpg6105+debug FLASH 613992 613992 0 0.0
RAM 99612 99612 0 0.0
stm32 light STM32WB5MM-DK FLASH 478472 478472 0 0.0
RAM 144716 144716 0 0.0
telink air-quality-sensor-app tlsr9528a_retention FLASH 620280 620280 0 0.0
RAM 50932 50932 0 0.0
all-clusters-app tlsr9118bdk40d FLASH 680860 680860 0 0.0
RAM 149544 149544 0 0.0
all-clusters-minimal-app tlsr9528a FLASH 774168 774168 0 0.0
RAM 110704 110704 0 0.0
bridge-app tlsr9258a FLASH 679010 679010 0 0.0
RAM 91588 91588 0 0.0
contact-sensor-app tlsr9528a_retention FLASH 621872 621872 0 0.0
RAM 50972 50972 0 0.0
light-switch-app-ota-shell-factory-data tlsr9528a FLASH 707762 707762 0 0.0
RAM 74312 74312 0 0.0
lighting-app-ota-factory-data tlsr9118bdk40d FLASH 620758 620758 0 0.0
RAM 145532 145532 0 0.0
lighting-app-ota-rpc-factory-data-4mb tlsr9518adk80d FLASH 806930 806930 0 0.0
RAM 99456 99456 0 0.0
lock-app-dfu tlsr9528a FLASH 654854 654854 0 0.0
RAM 67008 67008 0 0.0
ota-requestor-app tlsr9258a FLASH 697026 697026 0 0.0
RAM 91272 91272 0 0.0
pump-app-usb tlsr9518adk80d FLASH 631318 631318 0 0.0
RAM 55752 55752 0 0.0
pump-controller-app tlsr9518adk80d FLASH 608828 608828 0 0.0
RAM 52996 52996 0 0.0
shell tlsr9518adk80d FLASH 466810 466810 0 0.0
RAM 68660 68660 0 0.0
smoke_co_alarm-app tlsr9528a_retention FLASH 628662 628662 0 0.0
RAM 52692 52692 0 0.0
temperature-measurement-app-mars-ota tlsr9518adk80d FLASH 652698 652698 0 0.0
RAM 56632 56632 0 0.0
thermostat tlsr9518adk80d FLASH 633882 633882 0 0.0
RAM 53392 53392 0 0.0
window-covering tlsr9118bdk40d FLASH 522774 522774 0 0.0
RAM 98532 98532 0 0.0
tizen all-clusters-app arm unknown 4820 4820 0 0.0
FLASH 1701560 1701560 0 0.0
RAM 89300 89300 0 0.0
chip-tool-ubsan arm unknown 10252 10252 0 0.0
FLASH 17308510 17308510 0 0.0
RAM 7561280 7561280 0 0.0

Copy link

github-actions bot commented Sep 3, 2024

PR #34817: Size comparison from 3de6f04 to eadb9c9

Full report (77 builds for bl602, bl702, bl702l, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
platform target config section 3de6f04 eadb9c9 change % change
bl602 lighting-app bl602 FLASH 1279604 1279604 0 0.0
RAM 95904 95904 0 0.0
bl602+mfd FLASH 1293458 1293458 0 0.0
RAM 96048 96048 0 0.0
bl602+rpc FLASH 1318568 1318568 0 0.0
RAM 104328 104328 0 0.0
bl702 lighting-app bl702 FLASH 945190 945190 0 0.0
RAM 15209 15209 0 0.0
bl702+mfd FLASH 947334 947334 0 0.0
RAM 15353 15353 0 0.0
bl702+rpc FLASH 1040246 1040246 0 0.0
RAM 24221 24221 0 0.0
bl706-eth FLASH 648006 648006 0 0.0
RAM 25305 25305 0 0.0
bl706-wifi FLASH 895420 895420 0 0.0
RAM 14525 14525 0 0.0
bl702l lighting-app bl702l FLASH 961594 961594 0 0.0
RAM 16868 16868 0 0.0
bl702l+mfd FLASH 964216 964216 0 0.0
RAM 17012 17012 0 0.0
cyw30739 light CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 677257 677257 0 0.0
RAM 78324 78324 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 697117 697117 0 0.0
RAM 80956 80956 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 697117 697117 0 0.0
RAM 80956 80956 0 0.0
CYW930739M2EVB-02 unknown 2040 2040 0 0.0
FLASH 654045 654045 0 0.0
RAM 73392 73392 0 0.0
light-switch CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 611193 611193 0 0.0
RAM 71292 71292 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 630821 630821 0 0.0
RAM 73844 73844 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 630821 630821 0 0.0
RAM 73844 73844 0 0.0
lock CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 629233 629233 0 0.0
RAM 74308 74308 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 648941 648941 0 0.0
RAM 76860 76860 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 648941 648941 0 0.0
RAM 76860 76860 0 0.0
thermostat CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 603501 603501 0 0.0
RAM 68348 68348 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 623361 623361 0 0.0
RAM 70980 70980 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 623361 623361 0 0.0
RAM 70980 70980 0 0.0
efr32 lighting-app BRD4187C FLASH 935316 935316 0 0.0
RAM 135120 135120 0 0.0
lock-app BRD4338a FLASH 740764 740764 0 0.0
RAM 197184 197184 0 0.0
window-app BRD4187C FLASH 1015856 1015856 0 0.0
RAM 127036 127036 0 0.0
esp32 all-clusters-app c3devkit DRAM 94224 94224 0 0.0
FLASH 1540250 1540250 0 0.0
IRAM 82538 82538 0 0.0
m5stack DRAM 115160 115160 0 0.0
FLASH 1550698 1550698 0 0.0
IRAM 117039 117039 0 0.0
linux air-purifier-app debug unknown 4624 4624 0 0.0
FLASH 2763665 2763665 0 0.0
RAM 128536 128536 0 0.0
all-clusters-app debug unknown 5408 5408 0 0.0
FLASH 6078532 6078532 0 0.0
RAM 510536 510536 0 0.0
all-clusters-minimal-app debug unknown 5304 5304 0 0.0
FLASH 5402956 5402956 0 0.0
RAM 240376 240376 0 0.0
bridge-app debug unknown 5288 5288 0 0.0
FLASH 4732472 4732472 0 0.0
RAM 217368 217368 0 0.0
chip-tool debug unknown 5832 5832 0 0.0
FLASH 12651022 12651022 0 0.0
RAM 558354 558354 0 0.0
chip-tool-ipv6only arm64 unknown 20352 20352 0 0.0
FLASH 11322876 11322876 0 0.0
RAM 608424 608424 0 0.0
fabric-admin debug unknown 5664 5664 0 0.0
FLASH 10949835 10949835 0 0.0
RAM 557066 557066 0 0.0
fabric-bridge-app debug unknown 4520 4520 0 0.0
FLASH 4555270 4555270 0 0.0
RAM 204016 204016 0 0.0
lighting-app debug+rpc+ui unknown 5936 5936 0 0.0
FLASH 5684369 5684369 0 0.0
RAM 228184 228184 0 0.0
lock-app debug unknown 5224 5224 0 0.0
FLASH 4787074 4787074 0 0.0
RAM 203768 203768 0 0.0
ota-provider-app debug unknown 4600 4600 0 0.0
FLASH 4426612 4426612 0 0.0
RAM 197752 197752 0 0.0
ota-requestor-app debug unknown 4536 4536 0 0.0
FLASH 4565198 4565198 0 0.0
RAM 202288 202288 0 0.0
shell debug unknown 4176 4176 0 0.0
FLASH 3105277 3105277 0 0.0
RAM 159168 159168 0 0.0
thermostat-no-ble arm64 unknown 9296 9296 0 0.0
FLASH 4325900 4325900 0 0.0
RAM 241896 241896 0 0.0
tv-app debug unknown 5504 5504 0 0.0
FLASH 6011813 6011813 0 0.0
RAM 583192 583192 0 0.0
tv-casting-app debug unknown 5168 5168 0 0.0
FLASH 10854957 10854957 0 0.0
RAM 645768 645768 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 FLASH 915384 915384 0 0.0
RAM 142243 142243 0 0.0
nrf7002dk_nrf5340_cpuapp FLASH 886616 886616 0 0.0
RAM 140382 140382 0 0.0
all-clusters-minimal-app nrf52840dk_nrf52840 FLASH 846812 846812 0 0.0
RAM 141062 141062 0 0.0
nxp contact k32w0+release FLASH 577796 577796 0 0.0
RAM 70444 70444 0 0.0
k32w1+release FLASH 593248 593248 0 0.0
RAM 62712 62712 0 0.0
light k32w0+release FLASH 613416 613416 0 0.0
RAM 69908 69908 0 0.0
k32w1+release FLASH 676872 676872 0 0.0
RAM 47848 47848 0 0.0
psoc6 all-clusters cy8ckit_062s2_43012 FLASH 1647932 1647932 0 0.0
RAM 210944 210944 0 0.0
all-clusters-minimal cy8ckit_062s2_43012 FLASH 1551604 1551604 0 0.0
RAM 207656 207656 0 0.0
light cy8ckit_062s2_43012 FLASH 1470396 1470396 0 0.0
RAM 200752 200752 0 0.0
lock cy8ckit_062s2_43012 FLASH 1463636 1463636 0 0.0
RAM 225064 225064 0 0.0
qpg lighting-app qpg6105+debug FLASH 656812 656812 0 0.0
RAM 105152 105152 0 0.0
lock-app qpg6105+debug FLASH 614208 614208 0 0.0
RAM 99612 99612 0 0.0
stm32 light STM32WB5MM-DK FLASH 483216 483216 0 0.0
RAM 144740 144740 0 0.0
telink air-quality-sensor-app tlsr9528a_retention FLASH 620504 620504 0 0.0
RAM 50932 50932 0 0.0
all-clusters-app tlsr9118bdk40d FLASH 687434 687434 0 0.0
RAM 149576 149576 0 0.0
all-clusters-minimal-app tlsr9528a FLASH 779874 779874 0 0.0
RAM 110736 110736 0 0.0
bridge-app tlsr9258a FLASH 679496 679496 0 0.0
RAM 91588 91588 0 0.0
contact-sensor-app tlsr9528a_retention FLASH 622096 622096 0 0.0
RAM 50972 50972 0 0.0
light-switch-app-ota-shell-factory-data tlsr9528a FLASH 708106 708106 0 0.0
RAM 74312 74312 0 0.0
lighting-app-ota-factory-data tlsr9118bdk40d FLASH 625734 625734 0 0.0
RAM 145556 145556 0 0.0
lighting-app-ota-rpc-factory-data-4mb tlsr9518adk80d FLASH 811866 811866 0 0.0
RAM 99480 99480 0 0.0
lock-app-dfu tlsr9528a FLASH 655078 655078 0 0.0
RAM 67008 67008 0 0.0
ota-requestor-app tlsr9258a FLASH 697362 697362 0 0.0
RAM 91272 91272 0 0.0
pump-app-usb tlsr9518adk80d FLASH 631804 631804 0 0.0
RAM 55752 55752 0 0.0
pump-controller-app tlsr9518adk80d FLASH 609052 609052 0 0.0
RAM 52996 52996 0 0.0
shell tlsr9518adk80d FLASH 466890 466890 0 0.0
RAM 68660 68660 0 0.0
smoke_co_alarm-app tlsr9528a_retention FLASH 628882 628882 0 0.0
RAM 52692 52692 0 0.0
temperature-measurement-app-mars-ota tlsr9518adk80d FLASH 653034 653034 0 0.0
RAM 56632 56632 0 0.0
thermostat tlsr9518adk80d FLASH 635784 635784 0 0.0
RAM 53400 53400 0 0.0
window-covering tlsr9118bdk40d FLASH 523038 523038 0 0.0
RAM 98532 98532 0 0.0
tizen all-clusters-app arm unknown 4840 4840 0 0.0
FLASH 1724876 1724876 0 0.0
RAM 89500 89500 0 0.0
chip-tool-ubsan arm unknown 10272 10272 0 0.0
FLASH 17511166 17511166 0 0.0
RAM 7601324 7601324 0 0.0

Copy link
Contributor

@andy31415 andy31415 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes:

  • naming updates
  • do not use .value for enums as that is harder to debug when things fail

Copy link
Contributor

@andy31415 andy31415 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have made the changes manually since I understand Mathieu is OOO currently.

@mergify mergify bot merged commit 9b58d4c into project-chip:master Sep 6, 2024
68 checks passed
@mkardous-silabs mkardous-silabs deleted the feature/icdm_5.1 branch September 17, 2024 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants