Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement: Add test name before and after the test in console output #1962

Merged
merged 1 commit into from
Apr 27, 2024

Conversation

kosstennbl
Copy link
Collaborator

Add information of which test is starting to the console output
Add test name to test result console output

Issues:

Refs: #1958

No documentation update is needed.
Only outputs have been modified, functionality shouldn't be affected in any way.

Copy link
Collaborator

@HashNuke HashNuke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kosstennbl Have you tried running the automated tests to check if they pass?

Quite a lot of the automated tests depend on assertions which use the pass/fail message. They would have to be updated appropriately.

Screenshot below is an example:

CleanShot 2024-04-11 at 10 20 24@2x

@kosstennbl
Copy link
Collaborator Author

True, i forgot about that.
Would it be fine to replace expected regex messages like "PASSED: Resources output logs to stdout and stderr" with just "PASSED"?

@kosstennbl kosstennbl changed the title output: Add test name before and after the test Draft: output: Add test name before and after the test Apr 15, 2024
@kosstennbl
Copy link
Collaborator Author

Waits for merge of #1954
Then messages should be redesigned with [[^\n]*] inside regex

@agentpoyo
Copy link
Collaborator

Please create WIP or DRAFT PRs as DRAFT's when creating, this will prevent any accidental merges as "Draft" status will grey out the "merge" button until they're actually ready for review and merging.

@kosstennbl kosstennbl force-pushed the output-enchancement branch 2 times, most recently from 532b929 to 5a1512d Compare April 19, 2024 11:05
@HashNuke HashNuke marked this pull request as draft April 21, 2024 03:52
@HashNuke
Copy link
Collaborator

I've changed the status of this PR to draft to reflect the title.

@kosstennbl kosstennbl force-pushed the output-enchancement branch from 5a1512d to f450b81 Compare April 23, 2024 13:34
@kosstennbl kosstennbl changed the title Draft: output: Add test name before and after the test output: Add test name before and after the test Apr 23, 2024
@kosstennbl
Copy link
Collaborator Author

Ready to test, not a draft anymore.
@HashNuke

Btw, i can't change status from draft to open (probably insufficient permissions)

@HashNuke HashNuke marked this pull request as ready for review April 23, 2024 15:09
@taylor taylor changed the title output: Add test name before and after the test Enhancement: Add test name before and after the test in console output Apr 23, 2024
@taylor taylor added enhancement New feature or request usability labels Apr 23, 2024
Copy link
Collaborator

@taylor taylor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 lgtm

Note/Warning: the new regex in the specs using .* has had issues in the past with multiline PASS/FAIL output and incorrect results.

cc: @wavell @denverwilliams

@HashNuke
Copy link
Collaborator

@kosstennbl I held back on this PR because the main branch builds were broken.

I've made some fixes to the kubescape issues in the kubescape-version-check branch. The build is green. So the changes should be merged into main branch anytime soon. It might introduce some conflicts because some automated tests and specs were updated.

@kosstennbl
Copy link
Collaborator Author

@HashNuke
Ok, sure, I'll try to resolve conflicts as soon as possible

@HashNuke
Copy link
Collaborator

@kosstennbl Quick update - #2004 has been merged into main. This PR is now showing 2 conflicts.

@kosstennbl kosstennbl force-pushed the output-enchancement branch from f450b81 to a0daaa7 Compare April 25, 2024 18:09
@kosstennbl
Copy link
Collaborator Author

@HashNuke Rebase done, please run tests.

@HashNuke
Copy link
Collaborator

@kosstennbl There are two failing tags in the build - https://github.com/cnti-testcatalog/testsuite/actions/runs/8860425833/job/24331957328

  • platform:security - The spec for exposed_dashboard has been removed in main. It still exists in your branch.
  • task_runner

@HashNuke
Copy link
Collaborator

For anyone else curious what the output now looks like with this PR, I just ran the workload test for a sample CNF. Screenshot below. LGTM.

CleanShot 2024-04-27 at 21 20 13@2x

Ref: cnti-testcatalog#1958
Add information of which test is starting to the console output
Add test name to test result console output

Signed-off-by: Konstantin Yarovoy <[email protected]>
@kosstennbl kosstennbl force-pushed the output-enchancement branch from a0daaa7 to 4b43358 Compare April 27, 2024 16:03
@kosstennbl
Copy link
Collaborator Author

@HashNuke Fixed task_runner (checked localy - it passed), deleted exposed_dashboard.
Please retest

@HashNuke
Copy link
Collaborator

Build passing - https://github.com/cnti-testcatalog/testsuite/actions/runs/8861261394

Will wait for the main branch build to pass before merging this PR.

@HashNuke HashNuke merged commit 538aa3e into cnti-testcatalog:main Apr 27, 2024
109 of 188 checks passed
@kosstennbl kosstennbl deleted the output-enchancement branch April 29, 2024 08:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request usability
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

4 participants