pipenv lock broken in 11.x #1677

gsemet · 2018-03-12T10:50:44Z

Hello

pipenv update seems broken since 11.x version for packages on our internal pypi (Nexus). We force a "pre-version" for a package with:

pipenv install 'pbr==3.1.2.dev54'

that is only available on our internal Nexus.
The Pipfile is configured with

[[source]]

url = "http://nexus.server/repository/pypi/simple"
verify_ssl = false

This worked fine in pipenv < 11.

Now, it does not find it even if it is available (see in the error message) when generating the lock file:

$ pipenv install 'pbr==3.1.2.dev54'
Installing pbr==3.1.2.dev54…

Adding pbr==3.1.2.dev54 to Pipfile's [packages]…
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
Could not find a version that matches pbr==3.1.2.dev54,>=3.1.2.dev54
Tried: 0.5.2.5.g5b3e942, 3.1.2.dev54.linux-x86_64, 0.5.0, 0.5.1, 0.5.2, 0.5.4, 0.5.5, 0.5.6, 0.5.7, 0.5.8, 0.5.10, 0.5.11, 0.5.12, 0.5.13, 0.5.14, 0.5.15, 0.5.16, 0.5.17, 0.5.18, 0.5.19, 0.5.20, 0.5.21, 0.5.22, 0.5.23, 0.6, 0.7.0, 0.8.0, 0.8.1, 0.8.2, 0.9.0, 0.9.0, 0.10.0, 0.10.0, 0.10.1, 0.10.1, 0.10.2, 0.10.2, 0.10.3, 0.10.3, 0.10.4, 0.10.4, 0.10.5, 0.10.5, 0.10.6, 0.10.6, 0.10.7, 0.10.7, 0.10.8, 0.10.8, 0.11.0, 0.11.0, 0.11.1, 0.11.1, 1.0.0, 1.0.0, 1.0.1, 1.0.1, 1.1.0, 1.1.0, 1.1.1, 1.1.1, 1.2.0, 1.2.0, 1.3.0, 1.3.0, 1.4.0, 1.4.0, 1.5.0, 1.5.0, 1.6.0, 1.6.0, 1.7.0, 1.7.0, 1.8.0, 1.8.0, 1.8.1, 1.8.1, 1.9.0, 1.9.0, 1.9.1, 1.9.1, 1.10.0, 1.10.0, 2.0.0, 2.0.0, 2.1.0, 2.1.0, 3.0.0, 3.0.0, 3.0.1, 3.0.1, 3.1.0, 3.1.0, 3.1.1, 3.1.1, 3.1.2.dev54, 3.1.2.dev54

We clearly see that 3.1.2.dev54 is available in this list. I do not want to depend on any other preversion than this package.

The generated Pipfile look good. Only the lock failed:

[[source]]
url = "http://nexus.server/repository/pypi/simple"
verify_ssl = false

[packages]
pbr = "==3.1.2.dev54"

[dev-packages]

Even with --pre it does not work:

$ pipenv install --pre 'pbr==3.1.2.dev54'
Installing pbr==3.1.2.dev54…
Requirement already satisfied: pbr==3.1.2.dev54 in /home/.../python3.5/site-packages

Adding pbr==3.1.2.dev54 to Pipfile's [packages]…
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
Could not find a version that matches pbr==3.1.2.dev54,>=3.1.2.dev54
Tried: 0.5.2.5.g5b3e942, 3.1.2.dev54.linux-x86_64, 0.5.0, 0.5.1, 0.5.2, 0.5.4, 0.5.5, 0.5.6, 0.5.7, 0.5.8, 0.5.10, 0.5.11, 0.5.12, 0.5.13, 0.5.14, 0.5.15, 0.5.16, 0.5.17, 0.5.18, 0.5.19, 0.5.20, 0.5.21, 0.5.22, 0.5.23, 0.6, 0.7.0, 0.8.0, 0.8.1, 0.8.2, 0.9.0, 0.9.0, 0.10.0, 0.10.0, 0.10.1, 0.10.1, 0.10.2, 0.10.2, 0.10.3, 0.10.3, 0.10.4, 0.10.4, 0.10.5, 0.10.5, 0.10.6, 0.10.6, 0.10.7, 0.10.7, 0.10.8, 0.10.8, 0.11.0, 0.11.0, 0.11.1, 0.11.1, 1.0.0, 1.0.0, 1.0.1, 1.0.1, 1.1.0, 1.1.0, 1.1.1, 1.1.1, 1.2.0, 1.2.0, 1.3.0, 1.3.0, 1.4.0, 1.4.0, 1.5.0, 1.5.0, 1.6.0, 1.6.0, 1.7.0, 1.7.0, 1.8.0, 1.8.0, 1.8.1, 1.8.1, 1.9.0, 1.9.0, 1.9.1, 1.9.1, 1.10.0, 1.10.0, 2.0.0, 2.0.0, 2.1.0, 2.1.0, 3.0.0, 3.0.0, 3.0.1, 3.0.1, 3.1.0, 3.1.0, 3.1.1, 3.1.1, 3.1.2.dev54, 3.1.2.dev54

pipenv install --skip-lock does work, so this issue only seems to occur when generating the lock file.

$ pipenv --rm
$ pipenv install --skip-lock
Creating a virtualenv for this project…
Using /usr/bin/python3 (3.5.2) to create virtualenv…
⠋Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /home/.../bin/python3
Also creating executable in /home/.../bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /home/.../toto-KPovq1sx
Installing dependencies from Pipfile…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 2/2 — 00:00:02
To activate this project's virtualenv, run the following:
 $ pipenv shell

$ pipenv lock                                                                                                                                                                                                  
Locking [dev-packages] dependencies…
Locking [packages] dependencies…

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
Could not find a version that matches pbr==3.1.2.dev54,>=3.1.2.dev54
Tried: 0.5.2.5.g5b3e942, 3.1.2.dev54.linux-x86_64, 0.5.0, 0.5.1, 0.5.2, 0.5.4, 0.5.5, 0.5.6, 0.5.7, 0.5.8, 0.5.10, 0.5.11, 0.5.12, 0.5.13, 0.5.14, 0.5.15, 0.5.16, 0.5.17, 0.5.18, 0.5.19, 0.5.20, 0.5.21, 0.5.22, 0.5.23, 0.6, 0.7.0, 0.8.0, 0.8.1, 0.8.2, 0.9.0, 0.9.0, 0.10.0, 0.10.0, 0.10.1, 0.10.1, 0.10.2, 0.10.2, 0.10.3, 0.10.3, 0.10.4, 0.10.4, 0.10.5, 0.10.5, 0.10.6, 0.10.6, 0.10.7, 0.10.7, 0.10.8, 0.10.8, 0.11.0, 0.11.0, 0.11.1, 0.11.1, 1.0.0, 1.0.0, 1.0.1, 1.0.1, 1.1.0, 1.1.0, 1.1.1, 1.1.1, 1.2.0, 1.2.0, 1.3.0, 1.3.0, 1.4.0, 1.4.0, 1.5.0, 1.5.0, 1.6.0, 1.6.0, 1.7.0, 1.7.0, 1.8.0, 1.8.0, 1.8.1, 1.8.1, 1.9.0, 1.9.0, 1.9.1, 1.9.1, 1.10.0, 1.10.0, 2.0.0, 2.0.0, 2.1.0, 2.1.0, 3.0.0, 3.0.0, 3.0.1, 3.0.1, 3.1.0, 3.1.0, 3.1.1, 3.1.1, 3.1.2.dev54, 3.1.2.dev54

The text was updated successfully, but these errors were encountered:

techalchemy · 2018-03-12T12:16:14Z

https://www.python.org/dev/peps/pep-0440/#pre-release-spelling - we use the packaging version parser to do comparisons I believe, so if you’re not using a valid pre-release specifier the >= operator might not work. Caveat that I’m on mobile and might be lying, but any chance you can make the available version use .pre or .rc? This isn’t the first issue I’ve seen regarding nexus compatibility though, so it may be that nexus isn’t fully standardized to the PEP about PyPI or whatever

kennethreitz · 2018-03-12T12:52:37Z

This might be a valid regression.

kennethreitz · 2018-03-12T12:53:46Z

We made some changes recently to this code that greatly increases our capabilities for private PyPI servers.

mfwarren · 2018-03-12T13:02:37Z

I've just encountered this as well - after updating pipenv.

$ pipenv install eth-keys==0.1.0b4
Installing eth-keys==0.1.0b4…

Adding eth-keys==0.1.0b4 to Pipfile's [packages]…
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
Could not find a version that matches eth-keys<0.2.0,==0.1.0b4,>=0.1.0b3
Tried: 0.1.0b1, 0.1.0b1, 0.1.0b2, 0.1.0b2, 0.1.0b3, 0.1.0b3, 0.1.0b4, 0.1.0b4, 0.2.0b1, 0.2.0b1, 0.2.0b2, 0.2.0b2, 0.2.0b3, 0.2.0b3

tribals · 2018-03-12T13:06:04Z

Pipenv version: `'11.5.2'`

Pipenv location: `'/home/tosh/.local/lib/python3.6/site-packages/pipenv'`

Python location: `'/usr/bin/python3'`

Other Python installations in `PATH`:

  - `2.7`: `/home/tosh/.local/share/virtualenvs/tmp.YI0NPIlsPX-uYQ-I9WY/bin/python2.7`
  - `2.7`: `/home/tosh/.local/share/virtualenvs/tmp.YI0NPIlsPX-uYQ-I9WY/bin/python2.7`
  - `2.7`: `/usr/bin/python2.7`
  - `3.6`: `/usr/bin/python3.6m`
  - `3.6`: `/usr/bin/python3.6`

  - `2.7.14`: `/home/tosh/.local/share/virtualenvs/tmp.YI0NPIlsPX-uYQ-I9WY/bin/python`
  - `2.7.14`: `/usr/bin/python`
  - `2.7.14`: `/home/tosh/.local/share/virtualenvs/tmp.YI0NPIlsPX-uYQ-I9WY/bin/python2`
  - `2.7.14`: `/usr/bin/python2`
  - `3.6.4`: `/usr/bin/python3`

PEP 508 Information:

{'implementation_name': 'cpython',
'implementation_version': '3.6.4',
'os_name': 'posix',
'platform_machine': 'x86_64',
'platform_python_implementation': 'CPython',
'platform_release': '4.15.7-60.current',
'platform_system': 'Linux',
'platform_version': '#1 SMP PREEMPT Thu Mar 1 13:02:55 UTC 2018',
'python_full_version': '3.6.4',
'python_version': '3.6',
'sys_platform': 'linux'}


System environment variables:

  - `PYTHONUNBUFFERED`
  - `LC_MEASUREMENT`
  - `LC_PAPER`
  - `LC_MONETARY`
  - `XDG_MENU_PREFIX`
  - `LANG`
  - `GDM_LANG`
  - `HISTIGNORE`
  - `LESS`
  - `DISPLAY`
  - `OLDPWD`
  - `EDITOR`
  - `COLORTERM`
  - `DESKTOP_AUTOSTART_ID`
  - `XDG_VTNR`
  - `SSH_AUTH_SOCK`
  - `VIRTUAL_ENV`
  - `XDG_SESSION_ID`
  - `XDG_GREETER_DATA_DIR`
  - `USER`
  - `PAGER`
  - `DESKTOP_SESSION`
  - `PIP_PYTHON_PATH`
  - `QT_QPA_PLATFORMTHEME`
  - `PWD`
  - `INPUTRC`
  - `HOME`
  - `TMP`
  - `XDG_SESSION_TYPE`
  - `XDG_DATA_DIRS`
  - `PIPENV_ACTIVE`
  - `XDG_SESSION_DESKTOP`
  - `LC_NUMERIC`
  - `GTK_MODULES`
  - `VTE_VERSION`
  - `SHELL`
  - `TERM`
  - `GNOME_KEYRING_CONTROL`
  - `XDG_SEAT_PATH`
  - `QT_IM_MODULE`
  - `XMODIFIERS`
  - `XDG_CURRENT_DESKTOP`
  - `SHLVL`
  - `XDG_SEAT`
  - `WINDOWID`
  - `GDMSESSION`
  - `GNOME_DESKTOP_SESSION_ID`
  - `LOGNAME`
  - `DBUS_SESSION_BUS_ADDRESS`
  - `XDG_RUNTIME_DIR`
  - `XAUTHORITY`
  - `XDG_SESSION_PATH`
  - `XDG_CONFIG_DIRS`
  - `PATH`
  - `PS1`
  - `HISTSIZE`
  - `SESSION_MANAGER`
  - `LC_TIME`
  - `_`

Pipenv–specific environment variables:

 - `PIPENV_ACTIVE`: `1`

Debug–specific environment variables:

  - `PATH`: `/home/tosh/.local/share/virtualenvs/tmp.YI0NPIlsPX-uYQ-I9WY/bin:/home/tosh/.local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/snap/bin`
  - `SHELL`: `/bin/bash`
  - `EDITOR`: `nvim`
  - `LANG`: `en_US.utf8`
  - `PWD`: `/tmp/tmp.YI0NPIlsPX`
  - `VIRTUAL_ENV`: `/home/tosh/.local/share/virtualenvs/tmp.YI0NPIlsPX-uYQ-I9WY`


---------------------------

Contents of `Pipfile` ('/tmp/tmp.YI0NPIlsPX/Pipfile'):

```toml
[[source]]

url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"


[packages]

celery = "~=3.1"


[dev-packages]



[requires]

python_version = "2.7"

Contents of Pipfile.lock ('/tmp/tmp.YI0NPIlsPX/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "f4c70177c89354439b29248c18f2906ff6c09d397c5682bfa1d2aba03cb9a6c5"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "2.7"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.python.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "amqp": {
            "hashes": [
                "sha256:4e28d3ea61a64ae61830000c909662cb053642efddbe96503db0e7783a6ee85b",
                "sha256:cba1ace9d4ff6049b190d8b7991f9c1006b443a5238021aca96dd6ad2ac9da22"
            ],
            "version": "==2.2.2"
        },
        "billiard": {
            "hashes": [
                "sha256:1d7b22bdc47aa52841120fcd22a74ae4fc8c13e9d3935643098184f5788c3ce6",
                "sha256:abd9ce008c9a71ccde2c816f8daa36246e92a21e6a799831b887d88277187ecd"
            ],
            "version": "==3.5.0.3"
        },
        "celery": {
            "hashes": [
                "sha256:1954a224805f3835e5b6f5998ec9fe51db3413cc49e59fc720d314c7913427cf",
                "sha256:6ced63033bc663e60c992564954dbb5c84c43899f7f1a04b739957350f6b55f3"
            ],
            "version": "==3.1.25"
        },
        "kombu": {
            "hashes": [
                "sha256:01f0da9fe222a2183345004243d1518c0fbe5875955f1b24842f2d9c65709ade",
                "sha256:4249d9dd9dbf1fcec471d1c2def20653c9310dd1a217272d77e4844f9d5273cb"
            ],
            "version": "==4.1.0"
        },
        "pytz": {
            "hashes": [
                "sha256:07edfc3d4d2705a20a6e99d97f0c4b61c800b8232dc1c04d87e8554f130148dd",
                "sha256:410bcd1d6409026fbaa65d9ed33bf6dd8b1e94a499e32168acfc7b332e4095c0",
                "sha256:ba18e6a243b3625513d85239b3e49055a2f0318466e0b8a92b8fb8ca7ccdf55f"
            ],
            "version": "==2018.3"
        },
        "vine": {
            "hashes": [
                "sha256:52116d59bc45392af9fdd3b75ed98ae48a93e822cee21e5fda249105c59a7a72",
                "sha256:6849544be74ec3638e84d90bc1cf2e1e9224cc10d96cd4383ec3f69e9bce077b"
            ],
            "version": "==1.1.4"
        }
    },
    "develop": {}
}

celery==3.1.25
  - billiard [required: >=3.3.0.23,<3.4, installed: 3.5.0.3]
  - kombu [required: >=3.0.37,<3.1, installed: 4.1.0]
    - amqp [required: >=2.1.4,<3.0, installed: 2.2.2]
      - vine [required: >=1.1.3, installed: 1.1.4]
  - pytz [required: >dev, installed: 2018.3]

required: >=3.0.37,<3.1, installed: 4.1.0

Very nice.

kennethreitz · 2018-03-12T13:17:13Z

Will investigate.

kennethreitz · 2018-03-12T13:40:02Z

We have no way of reproducing this.

kennethreitz · 2018-03-12T13:40:58Z

@mfwarren I was just able to run that command successfully. Please try v11.5.3.

mfwarren · 2018-03-12T13:51:12Z

in a fresh virtualenv. I got the error on pipenv install web3

kennethreitz · 2018-03-12T15:05:56Z

Could not find a version that matches eth-account<1.0.0,>=0.1.0a2

kennethreitz · 2018-03-12T15:06:05Z

conflicting dependencies

kennethreitz · 2018-03-12T15:10:45Z

This was noted in the changelog: https://github.com/pypa/pipenv/blob/master/HISTORY.txt#L12

kennethreitz · 2018-03-12T15:10:56Z

e.g. this was an intended change

mfwarren · 2018-03-12T16:16:16Z

thanks Kenneth

gsemet · 2018-03-12T18:03:22Z

Same issue on 11.6. I guess I have a beginning of idea:

if only happens when ONE of the dependencies has a comparison operator (ex: >=3.1.2.dev54), even if all other forces to ==3.1.2.dev54
pipenv lock works if ALL packages declares ==3.1.2.dev54

I think there is a small precedence issue, where the expected behavor (I need to force to a preversion manually, and all other dependencies should be happy given their version compatibility operator)

In short, when pipenv finds 2 packages with dependency declared as >=3.1.2.dev54 and ==3.1.2.dev54, it kind of turn off the "use 3.1.2.dev54 anyway".

gsemet · 2018-03-12T18:59:17Z

If we do generate using "pipenv lock --pre", it does work.

gsemet changed the title ~~pipenv update broken in 11.x~~ pipenv lock broken in 11.x Mar 12, 2018

kennethreitz closed this as completed Mar 12, 2018

gsemet mentioned this issue Mar 13, 2018

--pre per–package #1696

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pipenv lock broken in 11.x #1677

pipenv lock broken in 11.x #1677

gsemet commented Mar 12, 2018 •

edited

Loading

techalchemy commented Mar 12, 2018

kennethreitz commented Mar 12, 2018

kennethreitz commented Mar 12, 2018

mfwarren commented Mar 12, 2018

tribals commented Mar 12, 2018

kennethreitz commented Mar 12, 2018

kennethreitz commented Mar 12, 2018

kennethreitz commented Mar 12, 2018

mfwarren commented Mar 12, 2018

kennethreitz commented Mar 12, 2018

kennethreitz commented Mar 12, 2018

kennethreitz commented Mar 12, 2018

kennethreitz commented Mar 12, 2018

mfwarren commented Mar 12, 2018

gsemet commented Mar 12, 2018 •

edited

Loading

gsemet commented Mar 12, 2018 •

edited

Loading

pipenv lock broken in 11.x #1677

pipenv lock broken in 11.x #1677

Comments

gsemet commented Mar 12, 2018 • edited Loading

techalchemy commented Mar 12, 2018

kennethreitz commented Mar 12, 2018

kennethreitz commented Mar 12, 2018

mfwarren commented Mar 12, 2018

tribals commented Mar 12, 2018

kennethreitz commented Mar 12, 2018

kennethreitz commented Mar 12, 2018

kennethreitz commented Mar 12, 2018

mfwarren commented Mar 12, 2018

kennethreitz commented Mar 12, 2018

kennethreitz commented Mar 12, 2018

kennethreitz commented Mar 12, 2018

kennethreitz commented Mar 12, 2018

mfwarren commented Mar 12, 2018

gsemet commented Mar 12, 2018 • edited Loading

gsemet commented Mar 12, 2018 • edited Loading

gsemet commented Mar 12, 2018 •

edited

Loading

gsemet commented Mar 12, 2018 •

edited

Loading

gsemet commented Mar 12, 2018 •

edited

Loading