Skip to content

Commit

Permalink
Add framework for device basic composition test (#27208)
Browse files Browse the repository at this point in the history
* Add basic framework for device basic composition test

Problem:
  - No test exists that validates global data model and
    system model rules related to device composition.
  - This makes verification patchy and can miss non-conformance.

Changes in this PR:
  - Introduce the TC_DeviceBasicComposition test with already a few
    important tests, such as validation of presence and content for
    all mandatory global attributes.
  - Some additional frameworking to report non-fatal problems in
    Python testing framework.
  - Ability to dump the topology of a device.

Testing done:
  - Ran the test against all clusters app in a bootstrapped environment
    - `scripts/build_python.sh -m platform -i build-env`
    - `. ./scripts/activate.sh`
    - `scripts/examples/gn_build_example.sh examples/all-clusters-app/linux out/debug/standalone chip_config_network_layer_ble=false`
    - Then in a different terminal:
     - `rm -f kvs1 && out/debug/standalone/chip-all-clusters-app --KVS kvs1`
    - Back in initial terminal:
     - `python3 src/python_testing/TC_DeviceBasicComposition.py --manual-code 34970112332 --string-arg dump_device_composition_path:dump_file`
  - Added the test to CI running against same

* Restyled by isort

* Fix lints

* Apply review comments

* Fix to work with Python 3.9

* Restyled by autopep8

* Fix CI

---------

Co-authored-by: Restyled.io <[email protected]>
Co-authored-by: [email protected] <[email protected]>
  • Loading branch information
3 people authored and pull[bot] committed Sep 14, 2023
1 parent b45e813 commit f6c59c9
Show file tree
Hide file tree
Showing 3 changed files with 512 additions and 1 deletion.
1 change: 1 addition & 0 deletions .github/workflows/tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -510,6 +510,7 @@ jobs:
run: |
scripts/run_in_build_env.sh './scripts/tests/run_python_test.py --app out/linux-x64-all-clusters-ipv6only-no-ble-no-wifi-tsan-clang-test/chip-all-clusters-app --factoryreset --script-args "--log-level INFO -t 3600 --disable-test ClusterObjectTests.TestTimedRequestTimeout"'
scripts/run_in_build_env.sh './scripts/tests/run_python_test.py --app out/linux-x64-all-clusters-ipv6only-no-ble-no-wifi-tsan-clang-test/chip-all-clusters-app --factoryreset --app-args "--discriminator 1234 --KVS kvs1 --trace_decode 1" --script "src/python_testing/TC_RR_1_1.py" --script-args "--storage-path admin_storage.json --commissioning-method on-network --discriminator 1234 --passcode 20202021"'
scripts/run_in_build_env.sh './scripts/tests/run_python_test.py --app out/linux-x64-all-clusters-ipv6only-no-ble-no-wifi-tsan-clang-test/chip-all-clusters-app --factoryreset --app-args "--discriminator 1234 --KVS kvs1 --trace_decode 1" --script "src/python_testing/TC_DeviceBasicComposition.py" --script-args "--storage-path admin_storage.json --manual-code 10054912339"'
scripts/run_in_build_env.sh './scripts/tests/run_python_test.py --app out/linux-x64-all-clusters-ipv6only-no-ble-no-wifi-tsan-clang-test/chip-all-clusters-app --factoryreset --app-args "--discriminator 1234 --KVS kvs1 --trace_decode 1" --script "src/python_testing/TC_SC_3_6.py" --script-args "--storage-path admin_storage.json --commissioning-method on-network --discriminator 1234 --passcode 20202021"'
scripts/run_in_build_env.sh './scripts/tests/run_python_test.py --app out/linux-x64-all-clusters-ipv6only-no-ble-no-wifi-tsan-clang-test/chip-all-clusters-app --factoryreset --app-args "--discriminator 1234 --KVS kvs1 --trace_decode 1" --script "src/python_testing/TC_DA_1_7.py" --script-args "--storage-path admin_storage.json --commissioning-method on-network --discriminator 1234 --passcode 20202021 --bool-arg allow_sdk_dac:true"'
scripts/run_in_build_env.sh './scripts/tests/run_python_test.py --app out/linux-x64-all-clusters-ipv6only-no-ble-no-wifi-tsan-clang-test/chip-all-clusters-app --factoryreset --app-args "--discriminator 1234 --KVS kvs1 --trace_decode 1 --enable-key 000102030405060708090a0b0c0d0e0f" --script "src/python_testing/TC_TestEventTrigger.py" --script-args "--storage-path admin_storage.json --commissioning-method on-network --discriminator 1234 --passcode 20202021 --bool-arg allow_sdk_dac:true"'
Expand Down
Loading

0 comments on commit f6c59c9

Please sign in to comment.