Strict mode(s) - change validation rules dynamically?

[goodboy]

As per pytest-dev/pytest#1821 @RonnyPfannschmidt has mentioned it's worth discussing if pluggy.PluginManager should have a strict mode where every single plugin which is loaded is verified against a spec instead of only performing this validation each time PluginManager.check_pending() is called.

In pytest check_pending() is only done once just prior to the pytest_cmdline_main(config=config) call meaning unverified hooks can be loaded any time afterwards.

This issue is to open discussion and brainstorming.

[nicoddemus]

Note that pytest-dev/pytest#1821 is about check_pending being called too early, so plugins registered using other mechanisms or from conftest.py files can't declare hooks.

[goodboy]

@nicoddemus I agree but that issue brought up the question (at least for me ) about how pluggy should support strict checking more generally. I do think that if you're looking at pluggy from a more "general purpose plugin system perspective" this kind of thing could be useful and maybe even moreso if you can configure "strictness" per hook?

I'm thinking mutiple cases in order of strictness:

• always require a hookspec for every loaded hookimpl
• only require a spec after some dedicated call onwards (like check_pending() but instead enables the validation permanently after some point - check_pending_forever())
• keep the single call check that check_pending() is
• allow for disabling validation - disable_validation()?

I also think pending is not clear and validate() or something similar is a better name.

Just some ideas.

[nicoddemus]

I think the current approach of explicitly calling check_pending at some point is good enough TBH. Perhaps in the future we might encounter some other use-cases where it makes sense to have additional methods of validation, but I think for now we are good.

[goodboy]

@nicoddemus agreed it's obviously working well :)

I'll close this for now.