Support wildcards in ignore-unused.

[samcunliffe]

Is your feature request related to a problem? Please describe.

I have a codebase with types stubs and pytest plugins as dev dependencies. When running fawltydeps they all appear as false positives (they're not imported directly in our codebase but they're providing pytest fixtures and typing support).

Now, I know I can individually ignore-unused in the configuration, but there are a lot of them. So I end up maintaining a rather long list of package names for this tool to ignore (list of pytest-x and types-y) in addition to the actual dependencies.

Describe the solution you'd like

I'm not sure if there's a better way, but something that comes to mind would be to support wildcards:

[tool.fawltydeps]
ignore_unused = ["pytest-*", "types-*]

fawltydeps . --ignore-unused="pytest-*" --ignore-unused="types-*"

Describe alternatives you've considered

This could maybe be automatically handled in the logic of the code, something like how you handle the "-stubs" suffix. But I'm not sure that's better.

Additional context

[jherland]

Thank you for your request!

I'm not opposed to adding wildcard support for ignore-unused, but I'd like to examine the motivation in more detail, first. For one, we certainly want to support type stubs well enough that ignore-unused should not be necessary for those.

I started looking at adding types-* handling to the same code that handles the -stubs suffix, but I quickly discovered that types-* packages should already be covered: because we're not matching on the name of the package itself, but rather on the existence of a FOO-stubs import name provided by the package (which is then matched against an import FOO in your code; as such the type stubs are not ignored by FawltyDeps, but rather matched against your imports). This *-stubs naming is mandated by https://typing.readthedocs.io/en/latest/spec/distributing.html#stub-only-packages, so I'm wondering why these come out as false positives in your case. A couple of control questions:

1. Are your types-* packages following the spec? If they're generally available, feel free to send me a list, and I can check them for you.
2. Are you importing the names that these stubs provide type information for? If you're somehow using type stubs for modules that you're not importing then that would also explain why they initially appear as unused.

Moving on to the pytest-* packages: These are not like type stubs, as they're not as directly related to 3rd-party modules/imports in your code. For example, there are plugins like pytest-celery that pair up with the celery package, but pytest-analyzer has nothing to do with the analyzer package.

I'd rather classify pytest plugins in our "tool" category, for which we already provide a default list here. (of course, as soon as you provide your own ignore-unused you override this list...)

Adding pytest-* to this list is probably a good idea.

But of course, in order to do so, we must in fact support wildcards, and I don't see why these should not be generally available to our users, as well. So let's add those. But let's also make sure that you don't need to use them in the first place 😄

[samcunliffe]

Yeah,

I'm not opposed to adding wildcard support for ignore-unused, but I'd like to examine the motivation in more detail, first. For one, we certainly want to support type stubs well enough that ignore-unused should not be necessary for those.

Sorry, I think my brain jumped to a solution.

---

As for types, I have the following flagged (the code under scrutiny is closed-source 😞 so I can't show you anything more useful)

- 'types-PyYAML'
- 'types-Pygments'
- 'types-colorama'
- 'types-pyOpenSSL'
- 'types-requests'
- 'types-setuptools'
- 'types-ujson'

I have (correctly) not had, e.g. types-python-dateutils, flagged and it is one of my explicit dependencies, I'm directly import ing dateutils. So the direct import stuff is working. The other types stubs are included as they're needed to stop mypy flagging. So... the tool is 'working' in that they're directly unused. Probably I should retract my claim "false positives" here. I just want to suppress a whole bunch.

---

For pytest plugins I have a small list:

- 'pyfakefs'
- 'pytest-cov'
- 'pytest-freezer'
- 'pytest-mock'

Which, as you say: are all plugins that aren't imported, they provide fixtures (or coverage reporting). For what it's worth we also have pytest-asyncio and that's correctly not flagged as we have import pytest_asyncio in our codebase.