-
-
Notifications
You must be signed in to change notification settings - Fork 320
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
classify tests and extend coverage #745
Conversation
Signed-off-by: clux <[email protected]>
Signed-off-by: clux <[email protected]>
Codecov Report
@@ Coverage Diff @@
## master #745 +/- ##
==========================================
+ Coverage 60.34% 70.19% +9.85%
==========================================
Files 52 53 +1
Lines 3051 3352 +301
==========================================
+ Hits 1841 2353 +512
+ Misses 1210 999 -211
Continue to review full report at Codecov.
|
might need to add this to actions, but testing first Signed-off-by: clux <[email protected]>
Signed-off-by: clux <[email protected]>
Signed-off-by: clux <[email protected]>
Signed-off-by: clux <[email protected]>
Signed-off-by: clux <[email protected]>
Signed-off-by: clux <[email protected]>
Signed-off-by: clux <[email protected]>
Signed-off-by: clux <[email protected]>
Signed-off-by: clux <[email protected]>
Signed-off-by: clux <[email protected]>
should push it to 70% Signed-off-by: clux <[email protected]>
this forces our integration folder to be moved to e2e as it is the heaviest type of test. but crucially it leaves a path for us to run our integration tests in a matrix against various clusters. Signed-off-by: clux <[email protected]>
Signed-off-by: clux <[email protected]>
Signed-off-by: clux <[email protected]>
Signed-off-by: clux <[email protected]>
Signed-off-by: clux <[email protected]>
Signed-off-by: clux <[email protected]>
Signed-off-by: clux <[email protected]>
Signed-off-by: clux <[email protected]>
Ok, perfect. I am ready to merge this if you are happy with it. |
This pr clarifies the distinction between our different types of tests and writes a policy on when to write a test for what. The key change for that is in the new
CONTRIBUTING.md
.Classification: integration reclassified as e2e. New integration is
cargo test --ignored
(using a liveClient
)New tests
Some tests imported and re-purposed from examples with new invariants:
This lets us improve the coverage without having to build the very expensive example setup with tarpaulin (and also lets us potentially run this class of tests more easily across clusters).
Coverage Improvements
Almost a 10% bump locally. After merge: (NB: it doesn't count derive very well, so it's still misleading).
kube-client/src/lib.rs
-> integration tests that need client and core (web sockets, pod creation, api usage, discovery)kube/src/lib.rs
-> integration tests that need a combination of runtime, derive, and client (crd usage, crd discovery)Also some more sporadic integration tests in runtime's eventrecorder.
Test Policy
We need a test policy and proof that we uphold a test policy in general for #737 and this should be sufficient for the policy, and pointing at a constant line/upward trend on coverage should be enough to prove it is upheld.