[FR] Proposal/Discussion: Minimising asymmetry between plugin and in-tree customisation/extension

[abravalheri]

What's the problem this feature will solve?

Setuptools is very flexible and provide lots of customisation/extension opportunities for plugin authors and (to some degree) end-users, which makes handy packages like setuptools-scm and setuptools-svn possible.

On the plugin author side, the main interface for such customisations is via entry-points, which include distutils.commands, setuptools.finalize_distribution_options, setuptools.file_finders , egg_info.writers (considering distutils.setup_keywords deprecated).

This list is likely to evolve in the future, for example with setuptools.sub_commands.build if #2899 gets accepted, or other possible entry-points/hooks as mentioned in #2220 and #2372.

On the end-user side, the main interface is via the setup function arguments, which (to my best knowledge) corresponds basically to cmdclass. As mentioned in #2591, cmdclass can be problematic, the users should not have to overwrite an entire command class to get the same benefits that a plugin could bring.

I understand that the idea is to migrate to a more declarative approach, instead of a procedural one based on setup.py, but even PEP 517 admits that some times the customisations are too project specific that do not justify creating a separated project for them.

My intention with this issue is to discuss some approaches that would minimise this asymmetry.

The advantages are not only on the end-user side. From the developers’ perspective it would be easier to add new features if they don’t have to implement/document 2 ways of exposing them.

Describe the solution you'd like

The main part of proposal is to give end-users a way of specifying “build-time only” entry-points.

In terms of integration of the current existing code, this could be achieved by creating a Distribution.iter_entry_points method that wraps pkg_resources.iter_entry_points and chains the locally specified "build-time only" entry-points with the conventional ones.

However there would be still necessary to find a way of allowing end-users to say which entry-points should be added in build-time. The following sections work as a brain-storm on how this could be done.

Option A: Rely on PEP 517-style “in-tree” backend wrapper

PEP 517 considers situations were a project-specific customisation can be implemented via a backend wrapper. setuptools.build_meta could expose a function to register “build-time only” entry-points. For example, consider the following hypothetical decorator:

#  _custom_build/backend.py
from setuptools import Command, build_meta

@build_meta.entry_point_for_build("setuptools.sub_commands.build", "build_js")
class BuildJs(Command):
    """Transpile JavaScript!"""
    …

__all__ = [‘build_meta’]

# pyproject.toml
[build-system]
requires = ["setuptools", "wheel"]
backend-path = ["_custom_build"]
build-backend = "backend:build_meta"

Pros: As discussed in #2854 and #2897, with the deprecation of setup_requires, creating a backend wrapper is currently the “blessed” way of specifying dynamic build requirements. Centralising all the custom Python code (hopefully small) needed to run the build in a single place is a nice thing™️.
Cons: This feels like replacing setup.py with a new Python file (although it is completely within the vision of PEP517).

EDIT: A PEP517-style in-tree wrapper could also rely on the config_settings argument for the backend hooks...

Option B: Use a custom option in setup.py

Imagine that a user could do:

# setup.py
…
class BuildJs(Command):

    …

setup(
    …
    entry_points_for_build = {
        "setuptools.sub_commands.build": {"build_js": BuildJs}
    }
)

Pros: There is no big change in terms of how users are used to do things.
Cons: If the user needs to dynamically add dependencies (e.g. let’s suppose they change with GPU or something similar), then 2 Python files will be needed for the build (setup.py and the backend wrapper).

Option C: Use a custom option in setup.cfg (or in the future pyproject.toml)

Imagine a user could do

# setup.cfg
[options.entry_points_for_build]
setuptools.sub_commands.build =
    build_js = _local_script_not_included_in_the_wheel:BuildJs

Or (when PEP 621 is adopted)

[tool.setuptools.entry_points_for_build."setuptools.sub_commands.build"]
build_js = "_local_script_not_included_in_the_wheel:BuildJs"

Pros: "Declarative" / configuration can be read without exec-ing a Python script
Cons:

• The users need a Python script where to implement these functions anyway…
• Need to update the config parsing code
• More configs to think about when transitioning to PEP621

Alternative Solutions

We can always combine all the options presented or even create new ones.

Additional context

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct

[abravalheri]

A PEP517-style in-tree wrapper could also rely on the config_settings argument for the backend hooks...

from setuptools import Command
from setuptools import build_meta as _orig
from setuptools.build_meta import *


class BuildJs(Command):
    ...


def get_requires_for_build_wheel(self, config_settings=None):
    config_settings = config_settings or {}
    config_settings.update(
        entry_points_for_build={
            "setuptools.sub_commands.build": {"build_js": BuildJs}
        }
    )
    return _orig.get_requires_for_build_wheel(config_settings)

Let's call this Option A.2

[rayzchen]

For option a.2 how would you include this file in source distributions but not binary distributions without using MANIFEST.in? And is there a way of just using this code in setup.py?

[abravalheri]

Hi @rayzchen, I think option a.2 would require MANIFEST.in or VCS plugin (e.g. setuptools-scm, setuptools-svn). You need a backend wrapper, but that means that the user needs to make sure the file is present in the sdist.

The backend wrapper approach is (as far as I understand) incompatible with setup.py (i.e., you cannot specify PEP 517 hooks in setup.py).

The approach using setup.py would be option B.