Advanced Test plan tutorial (infra) #1662
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pieqq
requested changes
Jan 8, 2025
Comment on lines
+3
to
+5
| ================= | ||
| Writing Test Plan | ||
| ================= |
There was a problem hiding this comment.
Suggested change
| ================= | |
| Writing Test Plan | |
| ================= | |
| =================== | |
| Writing A Test Plan | |
| =================== |
| ================= | ||
|
|
||
| This tutorial will guide you in writing a test plan to test the Network | ||
| connection in your machine. We will do this by re-using tests that are already |
There was a problem hiding this comment.
Suggested change
| connection in your machine. We will do this by re-using tests that are already | |
| connection on your machine. We will do this by re-using tests that are already |
| Inclusions | ||
| ========== | ||
|
|
||
| When we want a test plan to contain a test what we do in Checkbox is including |
There was a problem hiding this comment.
Suggested change
| When we want a test plan to contain a test what we do in Checkbox is including | |
| When we want a test plan to contain a test, what we do in Checkbox is including |
| ☑ : Test that the network speed is acceptable | ||
|
|
||
| Note how, as we previously saw, Checkbox automatically pulled the resource | ||
| job need. This operation, as we previously mention, is not the safe way to go |
There was a problem hiding this comment.
Suggested change
| job need. This operation, as we previously mention, is not the safe way to go | |
| job needed. This operation, as we previously mentioned, is not the safe way to go |
| ``providers/tutorial/units/test-plan.pxu`` | ||
|
|
||
| We now have a convenient container where to put all tests we previously | ||
| developed, let's include them in a new ``test plan``. |
There was a problem hiding this comment.
Suggested change
| developed, let's include them in a new ``test plan``. | |
| developed, let's include them in a new ``test plan``: |
| Excluding a test via ``exclude`` in the test plan is different from using | ||
| ``exclude`` in the launcher. If you use ``exclude`` in the launcher, you | ||
| are modifying the test plan, so it will not be accepted as a submission on | ||
| C3, while if you use ``exclude`` in a test plan, you are creating a new, |
There was a problem hiding this comment.
Suggested change
| C3, while if you use ``exclude`` in a test plan, you are creating a new, | |
| C3, whereas if you use ``exclude`` in a test plan, you are creating a new, |
Comment on lines
+345
to
+346
| jobs, you should most likely avoid doing that as you may inadvertently lose | ||
| more jobs (and time) than you were aiming for. Try to always precisely match |
There was a problem hiding this comment.
Suggested change
| jobs, you should most likely avoid doing that as you may inadvertently lose | |
| more jobs (and time) than you were aiming for. Try to always precisely match | |
| jobs, you should most likely avoid doing that as you may inadvertently deselect | |
| more jobs than you were aiming for. Try to always precisely match |
| jobs, you should most likely avoid doing that as you may inadvertently lose | ||
| more jobs (and time) than you were aiming for. Try to always precisely match | ||
| what you want to exclude, for templates, for example, use the template id | ||
| whenever you can instead of regex matching the generated id |
There was a problem hiding this comment.
Suggested change
| whenever you can instead of regex matching the generated id | |
| whenever you can instead of regex matching the generated id. |
Comment on lines
+382
to
+383
| The reason is that all tests in the ``submission-cert-automated`` are mandatory | ||
| includes |
There was a problem hiding this comment.
Suggested change
| The reason is that all tests in the ``submission-cert-automated`` are mandatory | |
| includes | |
| The reason is that all tests in the ``submission-cert-automated`` nested part are mandatory | |
| includes: they will be executed regardless of any other rule in your test plan. |
| bootstrap_include: | ||
| network_iface_info | ||
| include: | ||
| network_available_.* |
There was a problem hiding this comment.
You were using a template id in the previous test plan definition, and you switched to regex. Is this intentional?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This is an additional advanced tutorial about test plans. It should introduce all topics related to test plan writing and useful tips to work with them
Resolved issues
Fixes: CHECKBOX-760
Documentation
(this is documentation)
Tests
N/A