You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
After I use template-id, the order will be random, the test cases can't keep the order which is pre-defined in wireless-netplan-cert-automated and client-cert-desktop-24-04-automated.
The order is different in each time when I run the checkbox.
The ordering of the regex is not an intended behavior. We can not ensure that the tests are going to be executed in the order they are defined in the pxu.
Even with the regex syntax, if one of the tests fails, the order after retrying will probably be different.
We could implement some alphabetical ordering that keeps the dependency structure of the tests, or force them to keep the order in which they are defined (also keeping the dependency structure) but that's not a feature at the moment.
Bug Description
After I use template-id, the order will be random, the test cases can't keep the order which is pre-defined in wireless-netplan-cert-automated and client-cert-desktop-24-04-automated.
The order is different in each time when I run the checkbox.
I create two branch to check this issue, wireless_use_regex and wireless_use_template_id.
And the output :
Regex_output.zip
Template_id_output.zip
checkbox job order will be different in each time:
And there is no any wireless test job difference if using the regex.
To Reproduce
base
certification-client
Environment
Relevant log output
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: