Installing local wheel sometimes creates incorrect Pipfile.lock #2390

fairbanksg · 2018-06-20T13:49:55Z

In my project, I have four libraries where I have local patches. I have built wheels for all four, and then
run pipenv install <path to local wheel> for each.

This results in a Pipfile that correctly refers to the local wheels for all four.
However, the Pipfile.lock refers to the local wheel for two of them, but not for the other two.
This leads to a clean pipenv install installing the packages from pypi instead.

This is the pipenv.help output with all four of these libraries installed.

$ python -m pipenv.help output

Pipenv version: '2018.05.18'

Pipenv location: '/Users/greg/.pyenv/versions/2.7.12/lib/python2.7/site-packages/pipenv'

Python location: '/Users/greg/.pyenv/versions/2.7.12/bin/python'

Other Python installations in PATH:

2.7: /Users/greg/.pyenv/versions/2.7.12/bin/python2.7
2.7: /Users/greg/.pyenv/versions/2.7.12/bin/python2.7
2.7: /Users/greg/.pyenv/shims/python2.7
2.7: /usr/bin/python2.7
3.6: /Users/greg/.pyenv/shims/python3.6m
3.6: /Users/greg/.pyenv/shims/python3.6
2.7.12: /Users/greg/.pyenv/versions/2.7.12/bin/python
2.7.12: /Users/greg/.pyenv/shims/python
2.7.10: /usr/bin/python
2.7.12: /Users/greg/.pyenv/versions/2.7.12/bin/python2
2.7.12: /Users/greg/.pyenv/shims/python2
3.6.5: /Users/greg/.pyenv/shims/python3

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '0',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '17.5.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 17.5.0: Mon Mar  5 22:24:32 PST 2018; root:xnu-4570.51.1~1/RELEASE_X86_64',
 'python_full_version': '2.7.12',
 'python_version': '2.7',
 'sys_platform': 'darwin'}

System environment variables:

GOPATH
LESS
GOROOT
TERM_PROGRAM_VERSION
PYENV_DIR
LOGNAME
USER
HOME
PATH
PYENV_VIRTUALENV_INIT
TERM_PROGRAM
LANG
__CF_USER_TEXT_ENCODING
TERM
SHELL
PYENV_SHELL
SHLVL
LESS_TERMCAP_me
LESS_TERMCAP_md
LESS_TERMCAP_mb
PIPENV_VENV_IN_PROJECT
DISPLAY
RBENV_SHELL
PYENV_VERSION
EDITOR
GREP_COLOR
TERM_SESSION_ID
XPC_SERVICE_NAME
SSH_AUTH_SOCK
VISUAL
Apple_PubSub_Socket_Render
PYTHONDONTWRITEBYTECODE
PIP_PYTHON_PATH
LESS_TERMCAP_so
TMPDIR
BROWSER
LESS_TERMCAP_ue
LSCOLORS
PYENV_HOOK_PATH
PYENV_ROOT
LESS_TERMCAP_se
PWD
LESS_TERMCAP_us
GREP_OPTIONS
XPC_FLAGS
LS_COLORS
PAGER
NODENV_SHELL

Pipenv–specific environment variables:

PIPENV_VENV_IN_PROJECT: 1

Debug–specific environment variables:

PATH: /Users/greg/.pyenv/versions/2.7.12/bin:/usr/local/Cellar/pyenv/1.2.4/libexec:/Users/greg/google-cloud-sdk/bin:/Users/greg/.nodenv/shims:/Users/greg/.rbenv/shims:/usr/local/Cellar/pyenv-virtualenv/1.1.3/shims:/Users/greg/.pyenv/shims:/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Users/greg/dev/go/bin:/usr/local/opt/go/libexec/bin:/Users/greg/.cargo/bin:/Users/greg/.local/bin
SHELL: /usr/local/bin/zsh
EDITOR: vim
LANG: en_US.UTF-8
PWD: /Users/greg/dev/pipenv_wheel_bug

Contents of Pipfile ('/Users/greg/dev/pipenv_wheel_bug/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[dev-packages]

[packages]
GoogleAppEnginePipeline = {path = "./wheels/GoogleAppEnginePipeline-1.9.22.1-py2-none-any.whl"}
GoogleAppEngineMapReduce = {path = "./wheels/GoogleAppEngineMapReduce-1.9.21.0-py2-none-any.whl"}
python-intercom = {path = "./wheels/python_intercom-3.1.0-py2-none-any.whl"}
python-email-normalizer = {path = "./wheels/python_email_normalizer-0.1.4-py2-none-any.whl"}

[requires]
python_version = "2.7"

Contents of Pipfile.lock ('/Users/greg/dev/pipenv_wheel_bug/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "99bbedc14e61e2bb5af985d0d78a0822d1aabace6e147f9b827c038efa129354"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "2.7"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "certifi": {
            "hashes": [
                "sha256:13e698f54293db9f89122b0581843a782ad0934a4fe0172d2a980ba77fc61bb7",
                "sha256:9fa520c1bacfb634fa7af20a76bcbd3d5fb390481724c597da32c719a7dca4b0"
            ],
            "version": "==2018.4.16"
        },
        "chardet": {
            "hashes": [
                "sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae",
                "sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691"
            ],
            "version": "==3.0.4"
        },
        "dnspython": {
            "hashes": [
                "sha256:80f89881b402fc3b931a936111b43bcfe3abd8b0005d27e50e3c5fb59f7260f8"
            ],
            "version": "==1.13.0"
        },
        "funcsigs": {
            "hashes": [
                "sha256:330cc27ccbf7f1e992e69fef78261dc7c6569012cf397db8d3de0234e6c937ca",
                "sha256:a7bb0f2cf3a3fd1ab2732cb49eba4252c2af4240442415b4abce3b87022a8f50"
            ],
            "markers": "python_version < '3.3'",
            "version": "==1.0.2"
        },
        "googleappenginecloudstorageclient": {
            "hashes": [
                "sha256:b00ad9e7ca5a5524f07b6099b4c1e920cca9badc8e0d57cb6c1e2fb976356c95"
            ],
            "version": "==1.9.22.1"
        },
        "googleappenginemapreduce": {
            "hashes": [
                "sha256:80f8f26d3d2c6d59e590949c1e625935a2202218c516775761f01c1d54a542b8"
            ],
            "version": "==1.9.21.0"
        },
        "googleappenginepipeline": {
            "hashes": [
                "sha256:cef7f04172870db661c8ae26fc1417b1bb30bdcd0019b1cdd15c5c4cd3c0ae1c"
            ],
            "version": "==1.9.22.1"
        },
        "graphy": {
            "hashes": [
                "sha256:5fc4819608d3ff1fb9f896e529a1604de4dcbafe56a5f6adc23f029babadcb02"
            ],
            "version": "==1.0.0"
        },
        "idna": {
            "hashes": [
                "sha256:156a6814fb5ac1fc6850fb002e0852d56c0c8d2531923a51032d1b70760e186e",
                "sha256:684a38a6f903c1d71d6d5fac066b58d7768af4de2b832e426ec79c30daa94a16"
            ],
            "version": "==2.7"
        },
        "inflection": {
            "hashes": [
                "sha256:18ea7fb7a7d152853386523def08736aa8c32636b047ade55f7578c4edeb16ca"
            ],
            "version": "==0.3.1"
        },
        "mock": {
            "hashes": [
                "sha256:5ce3c71c5545b472da17b72268978914d0252980348636840bd34a00b5cc96c1",
                "sha256:b158b6df76edd239b8208d481dc46b6afd45a846b7812ff0ce58971cf5bc8bba"
            ],
            "version": "==2.0.0"
        },
        "mox": {
            "hashes": [
                "sha256:424ee725ee12652802b4e86571f816059b0d392401ceae70bf6487d65602cba9"
            ],
            "version": "==0.5.3"
        },
        "pbr": {
            "hashes": [
                "sha256:3747c6f017f2dc099986c325239661948f9f5176f6880d9fdef164cb664cd665",
                "sha256:a9c27eb8f0e24e786e544b2dbaedb729c9d8546342b5a6818d8eda098ad4340d"
            ],
            "version": "==4.0.4"
        },
        "python-email-normalizer": {
            "path": "./wheels/python_email_normalizer-0.1.4-py2-none-any.whl"
        },
        "python-intercom": {
            "path": "./wheels/python_intercom-3.1.0-py2-none-any.whl"
        },
        "pytz": {
            "hashes": [
                "sha256:65ae0c8101309c45772196b21b74c46b2e5d11b6275c45d251b150d5da334555",
                "sha256:c06425302f2cf668f1bba7a0a03f3c1d34d4ebeef2c72003da308b3947c7f749"
            ],
            "version": "==2018.4"
        },
        "requests": {
            "hashes": [
                "sha256:63b52e3c866428a224f97cab011de738c36aec0185aa91cfacd418b5d58911d1",
                "sha256:ec22d826a36ed72a7358ff3fe56cbd4ba69dd7a6718ffd450ff0e9df7a47ce6a"
            ],
            "version": "==2.19.1"
        },
        "simplejson": {
            "hashes": [
                "sha256:1bdd7c7c8c3ece26a251c835e73627a5f825b6ac1d16a68f190c8c29a3a4d4fe",
                "sha256:1e651e49b91024e615267fe800ad094c1174800ff06a3b29652f4a656dc51228",
                "sha256:2fa4eafab7cb4f900ce7739129cef0da1f25acfa89089540ce6241a15a61df78",
                "sha256:346ef48e38d202634ef5f8402d3e043984ef9f504f00b2275807cc8a01ae7d31",
                "sha256:446cc58ef7d8a4c5cb336d6893fd6aca1c22800207c18aa72f1496798e2aa6e8",
                "sha256:523d3df8dd6a366f8911ffdd9778d2e52d174f1b2ae867c543f9f200ab06595d",
                "sha256:60427da697809f2ec0a49b1c9146bf858722bd04fabef77b2b4fca0e883595ef",
                "sha256:7418a069e046df6afc9024076a154d35c4df432ad081f2a1cd57661c9dc95b51",
                "sha256:839ce3128375c4b6a08b4d20e67befe8fc0d6cf7ed1a1ee6117f035225368de8",
                "sha256:8b7644d71f8fb11088660775da0ab09151583939edeb840a53bdf2c5acf3b725",
                "sha256:95d19832c666c5942c57e67132138a9332aa84519919a97bd9bbcff1fce7cefd",
                "sha256:97502022f2fe5cf78580d5b0889030a62f46080508d32a0992ee7d7d107790b8",
                "sha256:a0bdc46d207edaa1db128be6f5b84e415aa033a854ed02d40256fc12d72ce0d7",
                "sha256:ad332f65d9551ceffc132d0a683f4ffd12e4bc7538681100190d577ced3473fb",
                "sha256:b5263de68cd0891ff4fe8ceb14e6382635ffb1be2ee0c8f7e664681679cd8163",
                "sha256:bb68f637d12dccf7ddeece29b4a5f27f15b768dff7f02c198d3d7640f9b8d2dc",
                "sha256:d6b38d952a3e90022287998928b12a77471b054455bcded9731dfb8371c47df8",
                "sha256:d7e1191ddccabcdc5b300d4469e0fbe69abda6d5e1fba37dad66f8a9913a9994",
                "sha256:e22f4bda9177893eb89a6a3cfc5335ae393690407b9b3407e86fe47c2a4adb1d",
                "sha256:f00fb192452506454ce7dba6de5a0d5386631e1d6cbc8dcb7e7d4b220bb13c06",
                "sha256:fae2430550b625ab2284110f4802ed1a1cae45f96871afbb014ee10f30a37fa3"
            ],
            "version": "==3.15.0"
        },
        "six": {
            "hashes": [
                "sha256:70e8a77beed4562e7f14fe23a786b54f6296e34344c23bc42f07b15018ff98e9",
                "sha256:832dc0e10feb1aa2c68dcc57dbb658f1c7e65b9b61af69048abc87a2db00a0eb"
            ],
            "version": "==1.11.0"
        },
        "urllib3": {
            "hashes": [
                "sha256:a68ac5e15e76e7e5dd2b8f94007233e01effe3e50e8daddf69acfd81cb686baf",
                "sha256:b5725a0bd4ba422ab0e66e89e030c806576753ea3ee08554382c14e685d117b5"
            ],
            "version": "==1.23"
        }
    },
    "develop": {}
}

And here is a more minimal case, with just one of the problematic wheels installed:

$ python -m pipenv.help output

Pipenv version: '2018.05.18'

Pipenv location: '/Users/greg/.pyenv/versions/2.7.12/lib/python2.7/site-packages/pipenv'

Python location: '/Users/greg/.pyenv/versions/2.7.12/bin/python'

Other Python installations in PATH:

2.7: /Users/greg/.pyenv/versions/2.7.12/bin/python2.7
2.7: /Users/greg/.pyenv/versions/2.7.12/bin/python2.7
2.7: /Users/greg/.pyenv/shims/python2.7
2.7: /usr/bin/python2.7
3.6: /Users/greg/.pyenv/shims/python3.6m
3.6: /Users/greg/.pyenv/shims/python3.6
2.7.12: /Users/greg/.pyenv/versions/2.7.12/bin/python
2.7.12: /Users/greg/.pyenv/shims/python
2.7.10: /usr/bin/python
2.7.12: /Users/greg/.pyenv/versions/2.7.12/bin/python2
2.7.12: /Users/greg/.pyenv/shims/python2
3.6.5: /Users/greg/.pyenv/shims/python3

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '0',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '17.5.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 17.5.0: Mon Mar  5 22:24:32 PST 2018; root:xnu-4570.51.1~1/RELEASE_X86_64',
 'python_full_version': '2.7.12',
 'python_version': '2.7',
 'sys_platform': 'darwin'}

System environment variables:

GOPATH
LESS
GOROOT
TERM_PROGRAM_VERSION
PYENV_DIR
LOGNAME
USER
HOME
PATH
PYENV_VIRTUALENV_INIT
TERM_PROGRAM
LANG
__CF_USER_TEXT_ENCODING
TERM
SHELL
PYENV_SHELL
SHLVL
LESS_TERMCAP_me
LESS_TERMCAP_md
LESS_TERMCAP_mb
PIPENV_VENV_IN_PROJECT
DISPLAY
RBENV_SHELL
PYENV_VERSION
EDITOR
GREP_COLOR
TERM_SESSION_ID
XPC_SERVICE_NAME
SSH_AUTH_SOCK
VISUAL
Apple_PubSub_Socket_Render
PYTHONDONTWRITEBYTECODE
PIP_PYTHON_PATH
LESS_TERMCAP_so
TMPDIR
BROWSER
LESS_TERMCAP_ue
LSCOLORS
PYENV_HOOK_PATH
PYENV_ROOT
LESS_TERMCAP_se
PWD
LESS_TERMCAP_us
GREP_OPTIONS
XPC_FLAGS
LS_COLORS
PAGER
NODENV_SHELL

Pipenv–specific environment variables:

PIPENV_VENV_IN_PROJECT: 1

Debug–specific environment variables:

PATH: /Users/greg/.pyenv/versions/2.7.12/bin:/usr/local/Cellar/pyenv/1.2.4/libexec:/Users/greg/google-cloud-sdk/bin:/Users/greg/.nodenv/shims:/Users/greg/.rbenv/shims:/usr/local/Cellar/pyenv-virtualenv/1.1.3/shims:/Users/greg/.pyenv/shims:/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Users/greg/dev/go/bin:/usr/local/opt/go/libexec/bin:/Users/greg/.cargo/bin:/Users/greg/.local/bin
SHELL: /usr/local/bin/zsh
EDITOR: vim
LANG: en_US.UTF-8
PWD: /Users/greg/dev/pipenv_wheel_bug

Contents of Pipfile ('/Users/greg/dev/pipenv_wheel_bug/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[dev-packages]

[packages]
GoogleAppEnginePipeline = {path = "./wheels/GoogleAppEnginePipeline-1.9.22.1-py2-none-any.whl"}

[requires]
python_version = "2.7"

Contents of Pipfile.lock ('/Users/greg/dev/pipenv_wheel_bug/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "b75a3e0a0b6e09350a16bdf67ddf1572457bc8824a1272353cb992b894a41c51"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "2.7"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "googleappenginecloudstorageclient": {
            "hashes": [
                "sha256:b00ad9e7ca5a5524f07b6099b4c1e920cca9badc8e0d57cb6c1e2fb976356c95"
            ],
            "version": "==1.9.22.1"
        },
        "googleappenginepipeline": {
            "hashes": [
                "sha256:cef7f04172870db661c8ae26fc1417b1bb30bdcd0019b1cdd15c5c4cd3c0ae1c"
            ],
            "version": "==1.9.22.1"
        }
    },
    "develop": {}
}

Expected result

Pipfile.lock should refer to the local wheel for all four of the installed libraries, not just two of them.

Actual result

Two of the four libraries are not specified as paths to local wheels in the Pipfile.lock.

Steps to replicate

Extract wheels.zip, which contains the wheels in question.
pipenv install ./wheels/GoogleAppEnginePipeline-1.9.22.1-py2-none-any.whl
Optionally, install the other 3 wheels for comparison, but installing this one wheel is enough to see the problem.

The text was updated successfully, but these errors were encountered:

fairbanksg · 2018-06-20T17:29:05Z

I started looking at the code to try and see what the difference between these might be, and noticed the code was substantially different on master. So I installed pipenv from the master branch.

The master branch is closer to working in this case, but has one problem.

Here is what I now get in my Pipfile.lock for this library. It has the wheel path now.

"googleappenginepipeline": {
    "hashes": [
        "sha256:cef7f04172870db661c8ae26fc1417b1bb30bdcd0019b1cdd15c5c4cd3c0ae1c"
    ],
    "path": "./wheels/GoogleAppEnginePipeline-1.9.22.1-py2-none-any.whl",
    "version": "==1.9.22.1"
}

But if I pipenv --rm and pipenv install again, I get an error about the hashes.

THESE PACKAGES DO NOT MATCH THE HASHES FROM Pipfile.lock!. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
    GoogleAppEnginePipeline==1.9.22.1 from file:///Users/greg/dev/pipenv_test/wheels/GoogleAppEnginePipeline-1.9.22.1-py2-none-any.whl (from -r /var/folders/nz/q34d4zd11hv25cgy64y8h3480000gn/T/pipenv-AVgbxP-requirements/pipenv-YqASI4-requirement.txt (line 1)):
        Expected sha256 cef7f04172870db661c8ae26fc1417b1bb30bdcd0019b1cdd15c5c4cd3c0ae1c
             Got        13210232486bf683d928aae1a2d9d97edada6d2d83063dc23b71fb6bbcee9874

I ran shasum on the wheel file, and got the 132102... value, not the hash value that had been recorded in the Pipfile.lock.

The good news is, this is the only thing stopping this from working, if I manually update the hash in the Pipfile.lock, the problem in the original report is resolved.

fairbanksg · 2018-06-20T17:34:15Z

One last thing, three of my wheels end up with hash values in the Pipfile.lock (and they are all different from the sha256 of the wheel files), but strangely, the python-email-normalizer wheel has an empty hashes list in the Pipfile.lock (so it installs fine).

uranusjr · 2018-06-20T18:08:11Z

Good news: This has been fixed! The next version will likely drop in the next few days, upgrade and it will work correctly :D

Closing so this can be identified as done and doesn’t confuse the release process.

fairbanksg · 2018-06-20T18:11:53Z

@uranusjr As I noted in #2390 (comment), even installing pipenv from the master branch gives me issues with the hash values for the wheels.

Should I open a new issue for that?

uranusjr · 2018-06-20T19:04:17Z

I opened #2394 for the hash issue.

uranusjr closed this as completed Jun 20, 2018

uranusjr mentioned this issue Jun 20, 2018

Locked info for local wheel should contain hash of the wheel file itself #2394

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Installing local wheel sometimes creates incorrect Pipfile.lock #2390

Installing local wheel sometimes creates incorrect Pipfile.lock #2390

fairbanksg commented Jun 20, 2018

fairbanksg commented Jun 20, 2018

fairbanksg commented Jun 20, 2018

uranusjr commented Jun 20, 2018

fairbanksg commented Jun 20, 2018

uranusjr commented Jun 20, 2018

Installing local wheel sometimes creates incorrect Pipfile.lock #2390

Installing local wheel sometimes creates incorrect Pipfile.lock #2390

Comments

fairbanksg commented Jun 20, 2018

Expected result

Actual result

Steps to replicate

fairbanksg commented Jun 20, 2018

fairbanksg commented Jun 20, 2018

uranusjr commented Jun 20, 2018

fairbanksg commented Jun 20, 2018

uranusjr commented Jun 20, 2018