pipenv uninstall --all-dev uninstalls shared default dependencies #3722

fernandrone · 2019-05-01T22:24:41Z

Issue description

pipenv uninstall --all-dev uninstalls shared default dependencies

Expected result

When running pipenv uninstall --all-dev I'd expect all dev dependencies and their dependencies to be removed, except those shared by default dependencies. Doing so breaks the application and forces me to run pipenv install.

Actual result

At least on one scenario that I observed, pipenv uninstall --all-dev will actually remove dev dependencies that are shared by default dependencies.

Steps to replicate

I'm using this sample application of mine to reproduce the behavior: https://github.com/fbcbarbosa/flask-chuck-norris

You'll notice that click is shared by black (a dev dependency) and Flask (a "default" dependency. When running pipenv uninstall --all-dev, click is also removed, breaking Flask.

$ pipenv install
...
$ pipenv graph
black==19.3b0
  - appdirs [required: Any, installed: 1.4.3]
  - attrs [required: >=18.1.0, installed: 19.1.0]
  - click [required: >=6.5, installed: 7.0]
  - toml [required: >=0.9.4, installed: 0.10.0]
Flask==1.0.2
  - click [required: >=5.1, installed: 7.0]
  - itsdangerous [required: >=0.24, installed: 1.1.0]
  - Jinja2 [required: >=2.10, installed: 2.10.1]
    - MarkupSafe [required: >=0.23, installed: 1.1.1]
  - Werkzeug [required: >=0.14, installed: 0.15.2]
pyjokes==0.5.0
pytest==4.4.1
  - atomicwrites [required: >=1.0, installed: 1.3.0]
  - attrs [required: >=17.4.0, installed: 19.1.0]
  - more-itertools [required: >=4.0.0, installed: 7.0.0]
  - pluggy [required: >=0.9, installed: 0.9.0]
  - py [required: >=1.5.0, installed: 1.8.0]
  - setuptools [required: Any, installed: 41.0.1]
  - six [required: >=1.10.0, installed: 1.12.0]
$ pipenv uninstall --all-dev
...
Removing black from Pipfile…
Uninstalling click…
Uninstalling Click-7.0:
  Successfully uninstalled Click-7.0

No package click to remove from Pipfile.
...
$ pipenv graph # graph says click is installed, but it isn't
Flask==1.0.2
  - click [required: >=5.1, installed: 7.0]
  - itsdangerous [required: >=0.24, installed: 1.1.0]
  - Jinja2 [required: >=2.10, installed: 2.10.1]
    - MarkupSafe [required: >=0.23, installed: 1.1.1]
  - Werkzeug [required: >=0.14, installed: 0.15.2]
pyjokes==0.5.0
$ pipenv shell
Launching subshell in virtual environment…
...
$ pip freeze
Flask==1.0.2
itsdangerous==1.1.0
Jinja2==2.10.1
MarkupSafe==1.1.1
pyjokes==0.5.0
Werkzeug==0.15.2

Running the application will throw an error, because click is not installed.

To tell the truth, the documentation is not clear on the expected behavior:

--all-dev             Un-install all package from [dev-packages].

Still, I find it unlikely that this is working as intended. Since in any reasonably sized project, dev dependencies and default dependencies will share some common dependencies, running pipenv uninstall --all-dev would always need to be followed by pipenv install.

My use case is that I want to run pipenv as part of my CI, and after running my tests I'd like to prune all dev dependencies before shipping my application - specifically, I'll be copying the pipenv environment inside a docker image.

To give another example, I'd expect it to work like npm prune --production.

$ pipenv --support

Pipenv version: '2018.11.26'

Pipenv location: '/usr/lib/python3.7/site-packages/pipenv'

Python location: '/usr/bin/python3'

Python installations found:

3.7.3: /home/walker/Workspace/src/github.com/fbcbarbosa/flask-chuck/.venv/bin/python3
3.7.3: /usr/bin/python3.7
3.7.3: /usr/bin/python3.7m
2.7.15: /usr/bin/python2

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.3',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.0.9-200.fc29.x86_64',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP Mon Apr 22 00:55:30 UTC 2019',
 'python_full_version': '3.7.3',
 'python_version': '3.7',
 'sys_platform': 'linux'}

System environment variables:

PIPENV_VENV_IN_PROJECT
NVM_DIR
LS_COLORS
TERMINATOR_UUID
XDG_MENU_PREFIX
MODULES_RUN_QUARANTINE
LANG
GDM_LANG
HISTCONTROL
LESS
DISPLAY
HOSTNAME
OLDPWD
SDKMAN_CANDIDATES_API
KUBECONFIG
EDITOR
COLORTERM
NVM_CD_FLAGS
FPATH
DESKTOP_AUTOSTART_ID
USERNAME
JAVA_HOME
KDEDIRS
XDG_VTNR
GIO_LAUNCHED_DESKTOP_FILE_PID
ZSH
SSH_AUTH_SOCK
XDG_SESSION_ID
MODULES_CMD
USER
ENV
PAGER
LSCOLORS
DESKTOP_SESSION
GOPATH
GRADLE_HOME
PWD
HOME
KOPS_CLUSTER_NAME
SSH_AGENT_PID
XDG_SESSION_TYPE
BASH_ENV
XDG_DATA_DIRS
TERMINATOR_DBUS_NAME
XDG_SESSION_DESKTOP
LOADEDMODULES
SDKMAN_DIR
TERMINATOR_DBUS_PATH
MAIL
ZSH_CUSTOM
WINDOWPATH
SHELL
VTE_VERSION
TERM
QT_IM_MODULE
XMODIFIERS
NVM_BIN
SDKMAN_CANDIDATES_DIR
XDG_CURRENT_DESKTOP
GIO_LAUNCHED_DESKTOP_FILE
KOPS_STATE_STORE
XDG_SEAT
SHLVL
MANPATH
MODULEPATH
GDMSESSION
LOGNAME
DBUS_SESSION_BUS_ADDRESS
XDG_RUNTIME_DIR
XAUTHORITY
MODULEPATH_modshare
GOBIN
PATH
SDKMAN_VERSION
DATASTORE_TYPE
MODULESHOME
HISTSIZE
SDKMAN_PLATFORM
SESSION_MANAGER
LESSOPEN
BASH_FUNC_module%%
BASH_FUNC__module_raw%%
BASH_FUNC_switchml%%
_
APPLICATION_INSIGHTS_NO_DIAGNOSTIC_CHANNEL
TERM_PROGRAM
TERM_PROGRAM_VERSION
PIP_DISABLE_PIP_VERSION_CHECK
PYTHONDONTWRITEBYTECODE
PIP_PYTHON_PATH
PIPENV_ACTIVE
LC_CTYPE
VIRTUAL_ENV
PS1
PIP_SHIMS_BASE_MODULE
PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

PIPENV_VENV_IN_PROJECT: true
PIPENV_ACTIVE: 1

Debug–specific environment variables:

PATH: /home/walker/Workspace/src/github.com/fbcbarbosa/flask-chuck/.venv/bin:/home/walker/.nvm/versions/node/v12.1.0/bin:/home/walker/.sdkman/candidates/java/current/bin:/home/walker/.sdkman/candidates/gradle/current/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/walker/bin:/var/lib/snapd/snap/bin:/home/walker/Workspace/bin:/home/walker/go/bin:/home/walker/Workspace/bin:/home/walker/go/bin:/home/walker/Workspace/bin:/home/walker/go/bin
SHELL: /bin/zsh
EDITOR: vim
LANG: en_US.UTF-8
PWD: /home/walker/Workspace/src/github.com/fbcbarbosa/flask-chuck
VIRTUAL_ENV: /home/walker/Workspace/src/github.com/fbcbarbosa/flask-chuck/.venv

Contents of Pipfile ('/home/walker/Workspace/src/github.com/fbcbarbosa/flask-chuck/Pipfile'):

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
flask = "==1.0.2"
pyjokes = "==0.5.0"

[requires]
python_version = "3.7.3"

Contents of Pipfile.lock ('/home/walker/Workspace/src/github.com/fbcbarbosa/flask-chuck/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "6a10557b3e2690522705a2875ca320334b9903a13e8bf0af2fb0273707afdbf0"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.7.3"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "click": {
            "hashes": [
                "sha256:2335065e6395b9e67ca716de5f7526736bfa6ceead690adf616d925bdc622b13",
                "sha256:5b94b49521f6456670fdb30cd82a4eca9412788a93fa6dd6df72c94d5a8ff2d7"
            ],
            "version": "==7.0"
        },
        "flask": {
            "hashes": [
                "sha256:2271c0070dbcb5275fad4a82e29f23ab92682dc45f9dfbc22c02ba9b9322ce48",
                "sha256:a080b744b7e345ccfcbc77954861cb05b3c63786e93f2b3875e0913d44b43f05"
            ],
            "index": "pypi",
            "version": "==1.0.2"
        },
        "itsdangerous": {
            "hashes": [
                "sha256:321b033d07f2a4136d3ec762eac9f16a10ccd60f53c0c91af90217ace7ba1f19",
                "sha256:b12271b2047cb23eeb98c8b5622e2e5c5e9abd9784a153e9d8ef9cb4dd09d749"
            ],
            "version": "==1.1.0"
        },
        "jinja2": {
            "hashes": [
                "sha256:065c4f02ebe7f7cf559e49ee5a95fb800a9e4528727aec6f24402a5374c65013",
                "sha256:14dd6caf1527abb21f08f86c784eac40853ba93edb79552aa1e4b8aef1b61c7b"
            ],
            "version": "==2.10.1"
        },
        "markupsafe": {
            "hashes": [
                "sha256:00bc623926325b26bb9605ae9eae8a215691f33cae5df11ca5424f06f2d1f473",
                "sha256:09027a7803a62ca78792ad89403b1b7a73a01c8cb65909cd876f7fcebd79b161",
                "sha256:09c4b7f37d6c648cb13f9230d847adf22f8171b1ccc4d5682398e77f40309235",
                "sha256:1027c282dad077d0bae18be6794e6b6b8c91d58ed8a8d89a89d59693b9131db5",
                "sha256:24982cc2533820871eba85ba648cd53d8623687ff11cbb805be4ff7b4c971aff",
                "sha256:29872e92839765e546828bb7754a68c418d927cd064fd4708fab9fe9c8bb116b",
                "sha256:43a55c2930bbc139570ac2452adf3d70cdbb3cfe5912c71cdce1c2c6bbd9c5d1",
                "sha256:46c99d2de99945ec5cb54f23c8cd5689f6d7177305ebff350a58ce5f8de1669e",
                "sha256:500d4957e52ddc3351cabf489e79c91c17f6e0899158447047588650b5e69183",
                "sha256:535f6fc4d397c1563d08b88e485c3496cf5784e927af890fb3c3aac7f933ec66",
                "sha256:62fe6c95e3ec8a7fad637b7f3d372c15ec1caa01ab47926cfdf7a75b40e0eac1",
                "sha256:6dd73240d2af64df90aa7c4e7481e23825ea70af4b4922f8ede5b9e35f78a3b1",
                "sha256:717ba8fe3ae9cc0006d7c451f0bb265ee07739daf76355d06366154ee68d221e",
                "sha256:79855e1c5b8da654cf486b830bd42c06e8780cea587384cf6545b7d9ac013a0b",
                "sha256:7c1699dfe0cf8ff607dbdcc1e9b9af1755371f92a68f706051cc8c37d447c905",
                "sha256:88e5fcfb52ee7b911e8bb6d6aa2fd21fbecc674eadd44118a9cc3863f938e735",
                "sha256:8defac2f2ccd6805ebf65f5eeb132adcf2ab57aa11fdf4c0dd5169a004710e7d",
                "sha256:98c7086708b163d425c67c7a91bad6e466bb99d797aa64f965e9d25c12111a5e",
                "sha256:9add70b36c5666a2ed02b43b335fe19002ee5235efd4b8a89bfcf9005bebac0d",
                "sha256:9bf40443012702a1d2070043cb6291650a0841ece432556f784f004937f0f32c",
                "sha256:ade5e387d2ad0d7ebf59146cc00c8044acbd863725f887353a10df825fc8ae21",
                "sha256:b00c1de48212e4cc9603895652c5c410df699856a2853135b3967591e4beebc2",
                "sha256:b1282f8c00509d99fef04d8ba936b156d419be841854fe901d8ae224c59f0be5",
                "sha256:b2051432115498d3562c084a49bba65d97cf251f5a331c64a12ee7e04dacc51b",
                "sha256:ba59edeaa2fc6114428f1637ffff42da1e311e29382d81b339c1817d37ec93c6",
                "sha256:c8716a48d94b06bb3b2524c2b77e055fb313aeb4ea620c8dd03a105574ba704f",
                "sha256:cd5df75523866410809ca100dc9681e301e3c27567cf498077e8551b6d20e42f",
                "sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7"
            ],
            "version": "==1.1.1"
        },
        "pyjokes": {
            "hashes": [
                "sha256:01b90474d5c889c21cae88f0d5fc8db1334b2891a16df75cbf9a0886bfdea653",
                "sha256:745299c99b7a4015251b4876ecd59dd62803f458a9f0540d036f0f6627dcc67e"
            ],
            "index": "pypi",
            "version": "==0.5.0"
        },
        "werkzeug": {
            "hashes": [
                "sha256:0a73e8bb2ff2feecfc5d56e6f458f5b99290ef34f565ffb2665801ff7de6af7a",
                "sha256:7fad9770a8778f9576693f0cc29c7dcc36964df916b83734f4431c0e612a7fbc"
            ],
            "version": "==0.15.2"
        }
    },
    "develop": {}
}

The text was updated successfully, but these errors were encountered:

fernandrone · 2019-05-01T22:26:51Z

Also, if this is confirmed to be an issue, I'd happily open a PR and contribute!

This is the first sane Python package management workflow I've found, and the first one that almost enables me to purge dev dependencies from the environment (almost due to this bug... at least it's not that bad, because I can always run pipenv install again and it will only reinstall missing shared dependencies).

jxltom · 2019-05-09T15:12:23Z

Currently pipenv just simply removes all packages in dev but not smart to detect the shared dependencies of default and dev packages. Personally I feels what you are proposing is quite reasonable.

techalchemy · 2019-05-20T06:37:58Z

Yeah I'd definitely merge a PR which improves the behavior but I think we'd need an accompanying addition to /peeps describing what exactly we expect here, I know I haven't spent too much time considering this because we aren't looking to add too many knobs so e just made it as simple as possible -- just remove the things

it does seem a bit obvious that we shouldn't remove the things that are also in the other section though...

…tall --all-dev`

…ninstall all-dev

Filters list of packages to uninstall for shared dependencies when uninstall is used with the --all-dev option. This prevents shared dependencies from being uninstalled or removed from the default section of the lockfile. Adds a test to ensure that shared dependencies are not uninstalled and not removed from the pipfile or lockfile.

cur33 · 2020-02-05T19:02:59Z

Bump—This is still an issue. Not too difficult to fix when it happens, but annoying and definitely unexpected.

TTMaZa · 2020-03-09T16:17:13Z

May I draw your attention to this behaviour:

pipenv install deprecation         # note: this will install pyparsing since its a dependency of deprecation
pipenv install --dev pyparsing 
pipenv uninstall --all-dev         # note: this will uninstall pyparsing even if it is in the [packages] section of the Pipfile!!!
pipenv sync                        # note: this will reinstall pyparsing

So to sumarize the problem from my view:

What not works: pipenv uninstall --all-dev. This removes things that are still needed 👎
What works: pipenv --rm && pipenv sync. This realy slow but rock solid 👍
What I'm looking for is something such as 'pipenv clean --keep-non-dev-only' or pipenv uninstall --all-dev-only. This would by rock solid and fast 👍 👍

Ponewor · 2020-06-03T08:07:20Z

IMHO it should not be just optional. Why would anyone want to remove those shared dependencies? I've made a PR with proposed solution.

jxltom added the Type: Enhancement 💡 This is a feature or enhancement request. label May 9, 2019

techalchemy added good first issue Issues suitable as a newcomer to get familiar with Pipenv! help wanted labels May 20, 2019

ianpittwood added a commit to ianpittwood/pipenv that referenced this issue Sep 6, 2019

pypa#3722 Add test for excluding shared packages during `pipenv unins…

098fd82

…tall --all-dev`

ianpittwood added a commit to ianpittwood/pipenv that referenced this issue Sep 7, 2019

pypa#3722 Check overlapping packages between default and dev during u…

3affc39

…ninstall all-dev

ianpittwood added a commit to ianpittwood/pipenv that referenced this issue Sep 7, 2019

pypa#3722 Finish implementing and debugging fix

403811f

ianpittwood mentioned this issue Sep 7, 2019

#3722 Exclude shared dependencies for --all-dev opt #3938

Closed

2 tasks

Ponewor mentioned this issue Jun 3, 2020

Fix --all-dev behaviour #4310

Merged

2 tasks

frostming closed this as completed in #4310 Jun 5, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pipenv uninstall --all-dev uninstalls shared default dependencies #3722

pipenv uninstall --all-dev uninstalls shared default dependencies #3722

fernandrone commented May 1, 2019 •

edited

Loading

fernandrone commented May 1, 2019

jxltom commented May 9, 2019

techalchemy commented May 20, 2019

cur33 commented Feb 5, 2020

TTMaZa commented Mar 9, 2020

Ponewor commented Jun 3, 2020

pipenv uninstall --all-dev uninstalls shared default dependencies #3722

pipenv uninstall --all-dev uninstalls shared default dependencies #3722

Comments

fernandrone commented May 1, 2019 • edited Loading

Issue description

Expected result

Actual result

Steps to replicate

fernandrone commented May 1, 2019

jxltom commented May 9, 2019

techalchemy commented May 20, 2019

cur33 commented Feb 5, 2020

TTMaZa commented Mar 9, 2020

Ponewor commented Jun 3, 2020

fernandrone commented May 1, 2019 •

edited

Loading