pipenv lock nondeterminism with environment markers (again)

[jfly]

I reported this bug about pipenv behaving non-deterministically a while ago: #4967. @matteius and I both thought that it was fixed by upgrading to a vendored pip 22.x, but I just checked, and it looks like the problem is back. Actually, it looks like maybe it never went away!

The setup is the same as before: add gevent and sqlalchemy to a Pipfile, and observe that repeated pipenv locks result in the different lockfiles.

Proof/demo

pipenv 2022.8.5

$ docker run $(docker build -q --build-arg=PIP_VERSION=22.2.2 --build-arg=PIPENV_VERSION=2022.8.5 https://github.com/jfly/2022-02-22-pipenv-nondeterminism.git#main)
attempt 1: greenlet markers="platform_python_implementation == 'CPython'"
attempt 2: greenlet markers="platform_python_implementation == 'CPython'"
attempt 3: greenlet markers="platform_python_implementation == 'CPython'"
attempt 4: greenlet markers="python_version >= '3' and platform_machine == 'aarch64' or (platform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32')))))"

pipenv 2022.7.24

$ docker run $(docker build -q --build-arg=PIP_VERSION=22.2.2 --build-arg=PIPENV_VERSION=2022.7.24 https://github.com/jfly/2022-02-22-pipenv-nondeterminism.git#main)
attempt 1: greenlet markers="platform_python_implementation == 'CPython'"
attempt 2: greenlet markers="python_version >= '3' and platform_machine == 'aarch64' or (platform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32')))))"
attempt 3: greenlet markers="platform_python_implementation == 'CPython'"

pipenv 2022.4.21

$ docker run $(docker build -q --build-arg=PIP_VERSION=22.2.2 --build-arg=PIPENV_VERSION=2022.4.21 https://github.com/jfly/2022-02-22-pipenv-nondeterminism.git#main)
attempt 1: greenlet markers="python_version >= '3' and (platform_machine == 'aarch64' or (platform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32'))))))"
attempt 2: greenlet markers="python_version >= '3' and platform_machine == 'aarch64' or (platform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32')))))"
attempt 3: greenlet markers="python_version >= '3' and platform_machine == 'aarch64' or (platform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32')))))"
attempt 4: greenlet markers="platform_python_implementation == 'CPython'"

pipenv 2022.1.8

The version of pipenv that this was originally reported for (#4967)

$ docker run $(docker build -q --build-arg=PIP_VERSION=22.2.2 --build-arg=PIPENV_VERSION=2022.1.8 https://github.com/jfly/2022-02-22-pipenv-nondeterminism.git#main)
attempt 1: greenlet markers="python_version >= '3' and (platform_machine == 'aarch64' or (platform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32'))))))"
attempt 2: greenlet markers="python_version >= '3' and platform_machine == 'aarch64' or (platform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32')))))"
attempt 3: greenlet markers="platform_python_implementation == 'CPython'"

[matteius]

Definitely still an issue -- I tired a few things tonight without avail. I wish this code were not so complicated.

[bakhtiary]

The point of divergence is when pipenv calls resolver.py in a separate process. The requirements are passed via the PIPENV_PACKAGES env var. Depending on the order in which the requirements are passed, different outcomes are observed.
We can replicate this with the following oneliner:

 diff <(PIPENV_PACKAGES='sqlalchemy -i https://pypi.org/simple\ngevent -i https://pypi.org/simple' python -m pipenv.resolver) <(PIPENV_PACKAGES='gevent -i https://pypi.org/simple\nsqlalchemy -i https://pypi.org/simple' python -m pipenv.resolver)

The quick fix is to make sure the order of the dependencies are the same.

Secondly, why are the markers different any way?

Thirdly, should the order of dependencies affect the outcome. This is a more philosophical question when we are working with np-complete problems. Should we sort the dependencies so that we get a particular result every time even if the original order of the dependencies are changed?

Edit: running the diff between the same ordered inputs gives the same output so my assumption was wrong:

diff <(PIPENV_PACKAGES='sqlalchemy -i https://pypi.org/simple\ngevent -i https://pypi.org/simple' python -m pipenv.resolver) <(PIPENV_PACKAGES='sqlalchemy -i https://pypi.org/simple\ngevent -i https://pypi.org/simple' python -m pipenv.resolver)

Note that you might have to run the command several times before seeing effects.

[bakhtiary]

gets fixed by: #5279

[bakhtiary]

the #5279 PR was accidentally closed. The new PR that fixes this is: #5286

[matteius]

I think we also need to sort the constraints. @dqkqd care to open a PR?