Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TC-DD-2.1, 2.2, 3.11–3.14] - Updated Manual Scripts and Test Steps to Align with Test Plan and VS Changes #37048

Open
wants to merge 14 commits into
base: master
Choose a base branch
from
Prev Previous commit
Next Next commit
Resolved errors
Rajashreekalmane committed Jan 13, 2025
commit a9d622fe172966aeba359b0d2abc86be4fedc9bb
100 changes: 50 additions & 50 deletions src/app/tests/suites/certification/Test_TC_DD_3_14.yaml
Original file line number Diff line number Diff line change
@@ -61,11 +61,11 @@ tests:

- label: "Step 1: Locate and scan/read the Commissionees QR code using DUT"
verification: |
Locate the Commissionee’s QR code and scan/read it using the DUT. Ensure the DUT successfully scans/reads the QR code.
Execute the following command on the TH:
Locate the Commissionee’s QR code and scan/read it using the DUT. Ensure the DUT successfully scans/reads the QR code.
Execute the following command on the TH:

./chip-all-clusters-app --version 0 --vendor-id 0xFFF1 --product-id 0x8001 --custom-flow 2 --capabilities 4 --discriminator 3840 --passcode 20202021
Expected logs on the TH:
./chip-all-clusters-app --version 0 --vendor-id 0xFFF1 --product-id 0x8001 --custom-flow 2 --capabilities 4 --discriminator 3840 --passcode 20202021
Expected logs on the TH:
[1657235470.970680][371041:371041] CHIP:DL: Device Configuration:
[1657235470.970772][371041:371041] CHIP:DL: Serial Number: (not set)
[1657235470.970811][371041:371041] CHIP:DL: Vendor Id: 65521 (0xFFF1)
@@ -79,18 +79,18 @@ tests:
[1657235470.971199][371041:371041] CHIP:SVR: SetupQRCode: [MT:-24J029Q00KA0648G00]
[1657235470.971258][371041:371041] CHIP:SVR: Copy/paste the below URL in a browser to see the QR Code:
[1657235470.971293][371041:371041] CHIP:SVR: https://dhrishi.github.io/connectedhomeip/qrcode.html?data=MT%3A-24J029Q00KA0648G00
Verify the QR code is scanned by DUT, chip-tool does not support physically scanning QR codes
Verify the QR code is scanned by DUT, chip-tool does not support physically scanning QR codes
disabled: true

- label:
"Step 2a: Version String: Using the QR code from Step 1, generate a
new QR code but substituting out the current Version String with an
invalid Version String (i.e. '010' or any non-zero 3-bit value)"
verification: |
Using the chip-tool, execute the following command to create a new QR code with an invalid version string ('010' or any non-zero 3-bit value):
Using the chip-tool, execute the following command to create a new QR code with an invalid version string ('010' or any non-zero 3-bit value):

Use chip-tool's "payload generate-qrcode" command to help generate the new, invalid QR code
./chip-tool payload generate-qrcode --existing-payload MT:-24J029Q00KA0648G00 --allow-invalid-payload 1 --version 2
Use chip-tool's "payload generate-qrcode" command to help generate the new, invalid QR code
./chip-tool payload generate-qrcode --existing-payload MT:-24J029Q00KA0648G00 --allow-invalid-payload 1 --version 2
Expected Logs on the TH:
[1657235626568] [31698:16839018] CHIP: [TOO] QR Code: MT:034J029Q00KA0648G00
disabled: true
@@ -115,13 +115,13 @@ tests:
OnNetwork discovery capability)"
PICS: MCORE.DD.DISCOVERY_BLE
verification: |
Modify the QR code from Step 1 by ensuring the Discovery Capability bit string excludes BLE and is set to OnNetwork discovery capability.
Use the following command to generate the QR code:
Modify the QR code from Step 1 by ensuring the Discovery Capability bit string excludes BLE and is set to OnNetwork discovery capability.
Use the following command to generate the QR code:

Use chip-tool's "payload generate-qrcode" command to help generate the new, invalid QR code
Use chip-tool's "payload generate-qrcode" command to help generate the new, invalid QR code

./chip-tool payload generate-qrcode --existing-payload MT:-24J029Q00KA0648G00 --allow-invalid-payload 1 --rendezvous 4
Expected Logs on the TH:
./chip-tool payload generate-qrcode --existing-payload MT:-24J029Q00KA0648G00 --allow-invalid-payload 1 --rendezvous 4
Expected Logs on the TH:

[1657235754393] [31702:16840391] CHIP: [TOO] QR Code: MT:-24J029Q00KA0648G00

@@ -150,9 +150,9 @@ tests:
to OnNetwork discovery capability)"
PICS: MCORE.DD.DISCOVERY_PAF
verification: |
WIFI - PAF feature is out of scope for V1.4
WIFI - PAF feature is out of scope for V1.4

[Verification command to be filled by NXP]
[Verification command to be filled by NXP]

User has a QR code generated to pass into DUT. Using the example provided above, the payload would be "MT:-24J029Q00KA0648G00".
$ ./out/all-clusters-app/chip-all-clusters-app --version 0 --vendor-id 0xFFF1 --product-id 0x8001 --custom-flow 2 --capabilities 4 --discriminator 3840 --passcode 20202021
@@ -174,8 +174,8 @@ tests:
- label: "Step 4b: Scan/read the QR code of the TH device using the DUT"
PICS: MCORE.DD.DISCOVERY_PAF
verification: |
WIFI - PAF feature is out of scope for V1.4
[Verification command to be filled by NXP]
WIFI - PAF feature is out of scope for V1.4
[Verification command to be filled by NXP]
disabled: true

- label:
@@ -188,10 +188,10 @@ tests:
66666666, 77777777, 88888888, 99999999, 12345678, 87654321"
verification: |
Using the QR code from Step 1, replace the Passcode component with each of the following invalid passcodes: 00000000, 11111111, 22222222, 33333333, 44444444, 55555555, 66666666, 77777777, 88888888, 99999999, 12345678, and 87654321.
For each invalid passcode, generate a unique QR code, resulting in the following payloads:
Execute the following command on the TH:
For each invalid passcode, generate a unique QR code, resulting in the following payloads:
Execute the following command on the TH:

./chip-tool payload generate-qrcode --existing-payload <MT:-24J029Q00KA0648G00> --allow-invalid-payload 1 --rendezvous 4
./chip-tool payload generate-qrcode --existing-payload <MT:-24J029Q00KA0648G00> --allow-invalid-payload 1 --rendezvous 4
"MT:-24J029Q00OC0000000" (00000000),
"MT:-24J029Q00KMSP0Z800" (11111111),
"MT:-24J029Q00GWID1WH00" (22222222),
@@ -211,34 +211,34 @@ tests:
"Step 4b: Scan each of the generated QR codes from the previous step
using DUT"
verification: |
Scan each of the 12 QR codes generated in the previous step using the DUT. The DUT should parse the QR code and identify the invalid passcode.
As expected, the DUT will terminate the commissioning process based on the invalid passcode. The DUT should reject the commission and not complete the pairing process. This termination will follow the DUT manufacturer’s specific instructions.

The following command should be run for each QR code:
Expected failure due to the invalid passcode for each of the generated QR codes:
Run command failure: ../../third_party/connectedhomeip/src/controller/SetUpCodePairer.cpp:50: CHIP Error 0x0000002F: Invalid argument

Example for the first QR code:

./chip-tool pairing code 1 MT:-24J029Q00OC0000000

Run command failure: ../../third_party/connectedhomeip/src/controller/SetUpCodePairer.cpp:50: CHIP Error 0x0000002F: Invalid argument

Similarly, repeat the command for the rest of the QR codes:

MT:-24J029Q00OC0000000
MT:-24J029Q00KMSP0Z800
MT:-24J029Q00GWID1WH00
MT:-24J029Q00C4912TQ00
MT:-24J029Q008E.Q2QZ00
MT:-24J029Q004ORE3N610
MT:-24J029Q000YH24KF10
MT:-24J029Q00Y58S4HO10
MT:-24J029Q00UF-F5EX10
MT:-24J029Q00QPQ36B420
MT:-24J029Q004QG46Y900
MT:-24J029Q00YX018EW10
Ensure that the error message CHIP Error 0x0000002F: Invalid argument appears for each attempt, confirming the DUT has rejected the commissioning.
Scan each of the 12 QR codes generated in the previous step using the DUT. The DUT should parse the QR code and identify the invalid passcode.
As expected, the DUT will terminate the commissioning process based on the invalid passcode. The DUT should reject the commission and not complete the pairing process. This termination will follow the DUT manufacturer’s specific instructions.

The following command should be run for each QR code:
Expected failure due to the invalid passcode for each of the generated QR codes:
Run command failure: ../../third_party/connectedhomeip/src/controller/SetUpCodePairer.cpp:50: CHIP Error 0x0000002F: Invalid argument

Example for the first QR code:

./chip-tool pairing code 1 MT:-24J029Q00OC0000000

Run command failure: ../../third_party/connectedhomeip/src/controller/SetUpCodePairer.cpp:50: CHIP Error 0x0000002F: Invalid argument

Similarly, repeat the command for the rest of the QR codes:

MT:-24J029Q00OC0000000
MT:-24J029Q00KMSP0Z800
MT:-24J029Q00GWID1WH00
MT:-24J029Q00C4912TQ00
MT:-24J029Q008E.Q2QZ00
MT:-24J029Q004ORE3N610
MT:-24J029Q000YH24KF10
MT:-24J029Q00Y58S4HO10
MT:-24J029Q00UF-F5EX10
MT:-24J029Q00QPQ36B420
MT:-24J029Q004QG46Y900
MT:-24J029Q00YX018EW10
Ensure that the error message CHIP Error 0x0000002F: Invalid argument appears for each attempt, confirming the DUT has rejected the commissioning.
disabled: true

- label:
@@ -247,8 +247,8 @@ tests:
that is not 'MT:' (i.e. Prefix='AB:')"
verification: |
Generate a new QR code by replacing the current prefix (MT:) with an invalid prefix (AB:) from step 1, resulting in the
payload AB:-24J029Q00KA0648G00, and ensure the QR code is correctly generated and ready to be passed into the DUT with no applicable TH or DUT logs.
AB:-24J029Q00KA0648G00
payload AB:-24J029Q00KA0648G00, and ensure the QR code is correctly generated and ready to be passed into the DUT with no applicable TH or DUT logs.
AB:-24J029Q00KA0648G00
disabled: true

- label: