Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add code coverage options #934

Merged
merged 2 commits into from
Sep 30, 2024

Conversation

jmartin-tech
Copy link
Collaborator

@jmartin-tech jmartin-tech commented Sep 26, 2024

Enables usage of the pytest-cov plugin to limit coverage number to project source code in the garak/ path.

Quality of life entry for exploring test coverage values. Restricting the output to library code without including the tests/ path requires configuration be place in a settings file so pyproject.toml is elected as already available in this project.

Output of coverage is similar to:

---------- coverage: platform darwin, python 3.12.4-final-0 ----------
Name                                            Stmts   Miss  Cover
-------------------------------------------------------------------
garak/__init__.py                                   3      0   100%
garak/__main__.py                                   7      7     0%
garak/_config.py                                  141     12    91%
garak/_plugins.py                                 238     32    87%
...
garak/resources/theme/__init__.py                   3      0   100%
-------------------------------------------------------------------
TOTAL                                            7744   3029    61%

Verification

  • Run with the following:
python -m pip install pytest-cov
python -m pytest tests --cov

Signed-off-by: Jeffrey Martin <[email protected]>
@jmartin-tech
Copy link
Collaborator Author

For now this does not add pytest-cov as a test dependency though I am open to expand this to have it become part of dev test requirements.

@leondz
Copy link
Collaborator

leondz commented Sep 28, 2024

Let's expand it

Copy link
Collaborator

@leondz leondz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw, what's a good hook for automation?

@jmartin-tech
Copy link
Collaborator Author

For automation, initially flags could be added to output coverage as html when running linux tests on main only.

@jmartin-tech jmartin-tech merged commit cc88954 into NVIDIA:main Sep 30, 2024
8 checks passed
@jmartin-tech jmartin-tech deleted the task/code-coverage-options branch September 30, 2024 13:15
@github-actions github-actions bot locked and limited conversation to collaborators Sep 30, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants