tests/e2e: dump extra checker info #315
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
found a bug, converting to draft for now |
willfindlay
changed the title
willfindlay
force-pushed
the
pr/willfindlay/e2e-framework-dump-improvements
branch
from
278cb18 to
a4cb9a0
Compare
|
Should be gucci now. |
Sometimes we might only want to compile a subset of tests during local development. This flag lets us do that. Defaults to ./... to keep the old behaviour. Signed-off-by: William Findlay <[email protected]>
Rather than creating the export dir after tests when dumping info, created it before-hand. This will enable us to also dump information during tests, which I'll be using in a subsequent commit in this series. As a bonus, keeping context in this way will prevent us from erroneously creating multiple export dirs (one for each testenv.Test). Signed-off-by: William Findlay <[email protected]>
It's extremely helpful to have a per-checker dump of json events, since this can be matched up 1:1 with the event sequence number in the event checker logs. That way we can compare an individual json event with the checker logs to look for matches and/or events that we expected to match but did not. Signed-off-by: William Findlay <[email protected]>
These functions enable grabbing the remaining unmatched event checks for a given checker before a call to .Finalize(). We can use this to print unmatched checks at the end of tests. Signed-off-by: William Findlay <[email protected]>
This commit causes the rpcchecker to dump its unmatched checks at the end as a JSON file. This should help us better debug which checks are actually unmatched and cross check them against the sequence of seen events. Signed-off-by: William Findlay <[email protected]>
Mixing klog and logrus here was causing problems since we weren't writing per-step info to the checker logs buffer and thus were missing info in checker dumps. Switch to the logrus logger here for now so that we can dump all the info we need to debug failures. Signed-off-by: William Findlay <[email protected]>
willfindlay
force-pushed
the
pr/willfindlay/e2e-framework-dump-improvements
branch
from
d978cb3 to
bde8d8e
Compare
kkourt
approved these changes
Aug 16, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR makes two critical additions to the e2e test export. In particular, we will now dump a list of all events encountered by the checker in order as well as all unmatched checks from the event checker in JSON format. Combined, these two additions will help us debug failing tests by enabling us to match events 1:1 with the checker logs. See individual commits.