centralize testing depends in setup.cfg extra depends #96
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We have them duplicated in tox.ini. @jwodder prefers to describe them explicitly
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #96 +/- ##
=======================================
Coverage 91.93% 91.93%
=======================================
Files 4 4
Lines 521 521
Branches 83 83
=======================================
Hits 479 479
Misses 25 25
Partials 17 17 ☔ View full report in Codecov by Sentry. |
jwodder
requested changes
Aug 23, 2024
| @@ -5,12 +5,8 @@ isolated_build = True | |||
| minversion = 3.3.0 | |||
|
|
|||
| [testenv] | |||
| deps = | |||
| dev: joblib @ git+https://github.com/joblib/joblib.git | |||
There was a problem hiding this comment.
This line needs to be kept so that it can be used by the py-dev job in .github/workflows/test.yml (added in #42).
There was a problem hiding this comment.
oh! So
- this line means that it defines
py-devtox environment with that extra dependency - as long as our CI passes, we can upgrade joblib to not be
~= 1.1but rather>= 1.1.0, <1.4ATM or even without upper limit and just react on breakage with dev joblib and if needed preemptively having it fixed etc?
There was a problem hiding this comment.
meanwhile -- reverted that change, force pushed cleaned up 2nd commit
There was a problem hiding this comment.
we can upgrade joblib to not be
~= 1.1but rather>= 1.1.0, <1.4
What would be the point of that? Note that ~= 1.1 means >= 1.1, <2.0, not >= 1.1, <1.2.
There was a problem hiding this comment.
you are right, I misunderstood it as <1.2.
meanwhile -- we turned CI green ;-)
…test Most recent debate on tox.ini vs extra_depends is at con/solidation#43 (comment) Although I appreciate the convenience of "tox" as an entry point for testing, I increasingly find no support for it to be "the" location for testing depends listing. Moreover I keep running into cases of needing to fish out test dependencies somewhere else (tox.ini, nox etc) which then can differ across projects, and also require me to adopt some avoidable runtime etc overhead from running those extra test shims whenever pytest is just good enough. My arguments for generally adopting an approach of specifying test dependencies in setup.{cfg,py} or other "generic" locations as optional are: - pytest, and its extentions are used (imported) inside the tests. I, in 99% of cases, do consider "tests" to be the part of the source code. I would not consider them part of the source code whenever there is an outside test battery which is developed/maintained independently of the source code. As such, I think that dependencies for tests, as part of the source code, should be listed alongside with dependencies for the build/installation/run time dependencies. Some distributions even do "import test" across entire source code distribution and thus tend to decide or to request exclusion from source distributions (IMHO the wrong step in most of the cases). Ref: dandi/dandi-schema#249 - It is unfortunate that there is no "standard" convention on how/where to specify such test requirements, so I think it is ok to adopt [test] as the general convention among our projects. - tox.ini looses NOTHING from using "extras". - tox.ini is the correct location to describe environments and dependncies for external to source code tools/modules, i.e those not imported explicitly anywhere in the source code. - with description of test dependencies alongside with the main dependencies would benefit downstream distribution (debian, conda, gentoo, etc) packagers since they do not need to fish around for what other dependencies to install/recommend/suggest for the package. - I do appreciate the fact that test dependencies alone are not sufficient to run the tests, but invocation of the pytest is standardized enough to just invoke it against the source code. (given dependencies are installed) That is e.g. how dh-python helper in Debian operates -- if pytest dependency announced, just run pytest automagically.
yarikoptic
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Extended reasoning for moving definition to setup.cfg is in the last commit: ATM f394f30 , cited here:
extras test vs tox.ini:
Most recent debate on tox.ini vs extra_depends is at
con/solidation#43 (comment)
Although I appreciate the convenience of "tox" as an entry point for testing, I
increasingly find no support for it to be "the" location for testing depends
listing. Moreover I keep running into cases of needing to fish out test
dependencies somewhere else (tox.ini, nox etc) which then can differ across
projects, and also require me to adopt some avoidable runtime etc overhead from
running those extra test shims whenever pytest is just good enough.
My arguments for generally adopting an approach of specifying test
dependencies in setup.{cfg,py} or other "generic" locations as optional are:
pytest, and its extentions are used (imported) inside the tests. I, in 99%
of cases, do consider "tests" to be the part of the source code. I would not
consider them part of the source code whenever there is an outside test
battery which is developed/maintained independently of the source code.
As such, I think that dependencies for tests, as part of the source code,
should be listed alongside with dependencies for the build/installation/run
time dependencies. Some distributions even do "import test" across entire
source code distribution and thus tend to decide or to request exclusion from
source distributions (IMHO the wrong step in most of the cases).
Ref: Exclude tests from wheel dandi/dandi-schema#249
It is unfortunate that there is no "standard" convention on how/where to
specify such test requirements, so I think it is ok to adopt [test] as
the general convention among our projects.
tox.ini looses NOTHING from using "extras".
tox.ini is the correct location to describe environments and dependncies
for external to source code tools/modules, i.e those not imported explicitly
anywhere in the source code.
with description of test dependencies alongside with the main dependencies
would benefit downstream distribution (debian, conda, gentoo, etc)
packagers since they do not need to fish around for what other dependencies
to install/recommend/suggest for the package.
I do appreciate the fact that test dependencies alone are not sufficient to
run the tests, but invocation of the pytest is standardized enough to just
invoke it against the source code. (given dependencies are installed)
That is e.g. how dh-python helper in Debian operates -- if pytest dependency
announced, just run pytest automagically.