installer: use venv and improve error handling

[abn]

This change drops the dependency on virtualenv for the installer,
replacing environment creation with the built-in venv module available
in Python >= 3.2.

In addition, this change also improves error handling for subprocess
commands.

The following behavioural changes are also introduced.

• An error log is written to a secure temporary file with stdout and
  tracebacks on error.
• If an existing virtual environment exists prior to installation, this
  is saved, and used for recovery if installation fails.

Resolves: #4093
Resolves: #4089
Resolves: #4195
Resolves: #4463

[abn]

cc: @sdispater

[sdispater]

@abn Won't that be an issue on Debian-based system which, as far as I recall, do not have the python-venv package installed?

[abn]

I thought this was now a built-in on there as well. Will check. We can do a fall back. But think now there is also install-virtualenv.

[neersighted]

I thought this was now a built-in on there as well. Will check. We can do a fall back. But think now there is also install-virtualenv.

Pretty sure this is a non-issue. The python3.7-venv package just contains the pyvenv-3.7 wrapper -- it looks like the venv module belongs to the libpython3.7-stdlib package: https://packages.debian.org/buster/amd64/libpython3.7-stdlib/filelist

[neersighted]

Looks good to me! I have some minor (very minor) nitpicks, but this nicely solves the issue I was having and eliminates the complexity of shelling out to virtualenv!

[neersighted]

I've rebased this/fixed the merge conflicts and implemented the suggestions I made before (discussed offline with @abn) -- this is ready to merge assuming the checks pass.

Since #4666 was landed in master this will be subject to full integration test coverage.

[neersighted]

I've rebased this onto master, and squashed my contributions. We're now testing green across the board, with only one weird hack/workaround (can't call Subprocess with a Path on Windows).

For posterity, we were struggling with macOS + Python 3.10 -- it ended up being an issue with blanking the environment. Omitting that eliminates a whole class of weirdness -- we can always revisit if needed.

[neersighted]

@abn Won't that be an issue on Debian-based system which, as far as I recall, do not have the python-venv package installed?

It turns out I was wrong and we do end up with an error -- however, since Ubuntu/Debian monkey-patch Python, it's quite readable and is not a blocker:

Retrieving Poetry metadata

# Welcome to Poetry!

This will download and install the latest version of Poetry,
a dependency and package manager for Python.

It will add the `poetry` command to Poetry's bin directory, located at:

/home/ubuntu/.local/bin

You can uninstall at any time by executing this script with the --uninstall option,
and these changes will be reverted.

Installing Poetry (1.1.11): Creating environment
The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt install python3.8-venv

You may need to use sudo with that command.  After installing the python3-venv
package, recreate your virtual environment.

Failing command: ['/home/ubuntu/.local/share/pypoetry/venv/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']

[abn]

I have a wip fix for that coming as soon as I fix merge conflicts from master.

We'll be falling back to a virtualenv zipapp.

[neersighted]

Looks good -- the best fusion of both our changes. I'd like to fix that redundant return cls(target) and merge this!

[github-actions]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.