Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: add e2e testing #12

Open
tony-go opened this issue Jul 2, 2022 · 2 comments
Open

test: add e2e testing #12

tony-go opened this issue Jul 2, 2022 · 2 comments
Labels
documentation Improvements or additions to documentation good first issue Good for newcomers

Comments

@tony-go
Copy link
Member

tony-go commented Jul 2, 2022

Context

We don't have a clear way to execute e2e testing. So basically, we create a new repository and link the dependency through npm link, then we could execute nsci run and make manual assertions.

Goal

Have a suite of abstracts to execute e2e tests.

Requirements

  • have a dedicated npm script to run them: npm run test:e2e
  • implement tests for basics scenarios
  • add a section in the CONTRIBUTING.md to help newcomers
@tony-go tony-go added documentation Improvements or additions to documentation good first issue Good for newcomers labels Jul 2, 2022
@tony-go
Copy link
Member Author

tony-go commented Jul 2, 2022

@antoine-coulon it should be nice if you could list basic scenarios

@antoine-coulon
Copy link
Member

Hello @tony-go,

I do agree that we should enforce the testing library but IMHO the most lacking part about testing is the frontend side. It's true that it would be great to start implementing e2e testing to also cover reporters' outputs and checking for a specific sets of packages (with various types of configuration) that everything is ok (including the integration with @nodesecure/scanner), but the real databases might change and the tests might would become flaky and inconsistent (as already seen in @nodesecure/vulnera for instance). This already had some minor consequences for our only reporter (at the moment) which is the console #8 but this was only a "reporting" bug as the whole logic behind is mostly covered up.

About that UI testing, the main concern for me is probably the tools for testing any type of output provided by the @nodesecure/ci lib. For now, only a console reporter has been developed and then we could imagine some HTML or any other type of reporter requiring different types of testing tools.

It could be interesting to first list and discuss about tools that could help us on that subject especially on the console reporter so that we could perform high level testing on the UI side in the same spirit as React Testing Library.

For the scenario parts, it would be only a matter of providing various sets of interpreted payloads and asserting that the reporting output is correct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants