Using newer Python versions (>= 3.12) for pythonBuild

[4KevR]

Hi everyone,

I am working with the pythonBuild step from this library and want to migrate to using Python 3.13 for the build. The issue is that setuptools is not available in a "clean" Python 3.12 (and also 3.13) onwards; therefore, the Docker image python:3.13 cannot be used. I am not aware of any entry point to configure package installations before the Python build is executed. This would be the relevant code part:

jenkins-library/cmd/pythonBuild.go

Line 115 in e927605

err := utils.RunExecutable(virutalEnvironmentPathMap["python"], flags...)

From my understanding, it would be necessary to install a build backend (like setuptools) beforehand to make this execution successful. Are there other ways of ensuring that a build backend is available, or could we extend the code to handle the build for newer Python versions?

I also have another question: Are there plans to support the newer pyproject.toml file for setup (which also has the possibility to specify a build backend), or will only the setup.py approach be available in the future?

Best regards and thank you,
Kevin

[PrimeF]

Hi all,

chiming in and supporting the request as well, especially because the currently supported way of building Python packages, namely via the setup.py commands, is deprecated and according to the documentation "MUST NOT be run anymore" (see https://packaging.python.org/en/latest/guides/modernize-setup-py-project/#should-setup-py-be-deleted).

Instead the command should be exchanged for: python -m build.

Would appreciate your support on this.

Thanks!
Bests,
Fabrizio

[PrimeF]

Adding to the findings above, it looks like the created Python virtual environment is not sourced properly.

jenkins-library/cmd/pythonBuild.go

Line 128 in 85dfe3e

err = utils.RunExecutable("bash", "-c", "source "+filepath.Join(config.VirutalEnvironmentName, "bin", "activate"))

Indeed, by running bash -c, the sourcing occurs in a sub-shell, so that the subsequent Python commands do not use the virtual environment Python executable, but rather the system-wide one.

One option for a workaround could be to "inject" the required commands within the buildFlags of the pythonBuild stage.

steps:
  pythonBuild:
    buildFlags:
      [
        "-c",
        "import subprocess; subprocess.check_call('. piperBuild-env/bin/activate && python -m pip install build && python -m build', shell=True)",
      ]
    virtualEnvironmentName: "piperBuild-env"
    dockerImage: python:3.12

Currently the build is executed via the command python <custom buildFlags> setup.py sdist bdist_wheel:

jenkins-library/cmd/pythonBuild.go

Line 115 in 85dfe3e

err := utils.RunExecutable(virutalEnvironmentPathMap["python"], flags...)

Hence, by injecting the -c option it is possible to run custom shell commands from within Python code.

Not sure about the security implications of this workaround as well as possible side effects though.