extras not installed for file-based dependencies

[petereon]

Issue description

When I am installing from a remote .whl file, the dependencies under specified extras do not get pulled.

I have managed to reproduce this with a few packages, I am providing an example with FastAPI.

Steps to replicate

I have executed the following:

pipenv install 'fastapi[dev]@https://files.pythonhosted.org/packages/4e/1a/04887c641b67e6649bde845b9a631f73a7abfbe3afda83957e09b95d88eb/fastapi-0.95.2-py3-none-any.whl'

Relevant excerpt from .whl file METADATA:

Provides-Extra: dev
Requires-Dist: pre-commit<3.0.0,>=2.17.0; extra == 'dev'
Requires-Dist: ruff==0.0.138; extra == 'dev'
Requires-Dist: uvicorn[standard]<0.21.0,>=0.12.0; extra == 'dev'

Expected result

ruff, pre-commit and uvicorn[standard] should be installed and included in the Pipfile.lock

Actual result

Packages are not installed nor included in the Pipfile.lock

pipenv install output:

$ pipenv install 'fastapi[dev]@https://files.pythonhosted.org/packages/4e/1a/04887c641b67e6649bde845b9a631f73a7abfbe3afda83957e09b95d88eb/fastapi-0.95.2-py3-none-any.whl' --
verbose
Installing fastapi[dev]@https://files.pythonhosted.org/packages/4e/1a/04887c641b67e6649bde845b9a631f73a7abfbe3afda83957e09b95d88eb/fastapi-0.95.2-py3-none-any.whl...
Resolving fastapi@https://files.pythonhosted.org/packages/4e/1a/04887c641b67e6649bde845b9a631f73a7abfbe3afda83957e09b95d88eb/fastapi-0.95.2-py3-none-any.whl...
/home/petervyboch/.local/pipx/venvs/pipenv/lib64/python3.9/site-packages/pipenv/patched/pip/_internal/models/link.py:391: PipDeprecationWarning: DEPRECATION: 
https://files.pythonhosted.org/packages/4e/1a/04887c641b67e6649bde845b9a631f73a7abfbe3afda83957e09b95d88eb/fastapi-0.95.2-py3-none-any.whl#egg=fastapi[dev] contains an egg 
fragment with a non-PEP 508 name pip 25.0 will enforce this behaviour change. A possible replacement is to use the req @ url syntax, and remove the egg fragment. Discussion 
can be found at https://github.com/pypa/pip/issues/11617
  deprecated(
Installing...
Installing package: https://files.pythonhosted.org/packages/4e/1a/04887c641b67e6649bde845b9a631f73a7abfbe3afda83957e09b95d88eb/fastapi-0.95.2-py3-none-any.whl#egg=fastapi
⠋ Installing fastapi...Writing supplied requirement line to temporary file: 'https://files.pythonhosted.org/packages/4e/1a/04887c641b67e6649bde845b9a631f73a7abfbe3afda83957e09b95d88eb/fastapi-0.95.2-py3-none-any.whl#egg=fastapi[dev]'
Installing 'fastapi'
$ /home/petervyboch/.local/share/virtualenvs/pipenv-testing1-kBvEswyq/bin/python /home/petervyboch/.local/pipx/venvs/pipenv/lib/python3.9/site-packages/pipenv/patched/pip/__pip-runner__.py install --no-input --verbose --upgrade --exists-action=i -r /tmp/pipenv-ub_udvhk-requirements/pipenv-1ayszplz-requirement.txt -i https://pypi.org/simple
Using source directory: '/home/petervyboch/.local/share/virtualenvs/pipenv-testing1-kBvEswyq/src'
Adding fastapi to Pipfile's [packages] ...
✔ Installation Succeeded
Installing dependencies from Pipfile.lock (d144dc)...
Writing supplied requirement line to temporary file: 'https://files.pythonhosted.org/packages/4e/1a/04887c641b67e6649bde845b9a631f73a7abfbe3afda83957e09b95d88eb/fastapi-0.95.2-py3-none-any.whl#egg=fastapi --hash=sha256:d374dbc4ef2ad9b803899bd3360d34c534adc574546e25314ab72c0c4411749f'
Install Phase: Standard Requirements
Preparing Installation of 'fastapi'
$ /home/petervyboch/.local/share/virtualenvs/pipenv-testing1-kBvEswyq/bin/python /home/petervyboch/.local/pipx/venvs/pipenv/lib/python3.9/site-packages/pipenv/patched/pip/__pip-runner__.py install -i https://pypi.org/simple --no-input --upgrade --no-deps --exists-action=i -r /tmp/pipenv-ub_udvhk-requirements/pipenv-t13ii_g9-hashed-reqs.txt
Using source directory: '/home/petervyboch/.local/share/virtualenvs/pipenv-testing1-kBvEswyq/src'

Collecting fastapi==0.95.2 (from -r /tmp/pipenv-ub_udvhk-requirements/pipenv-t13ii_g9-hashed-reqs.txt (line 1))

  Using cached fastapi-0.95.2-py3-none-any.whl (56 kB)


To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.

Pipfile.lock:

{
    "_meta": {
        "hash": {
            "sha256": "894b2e7adcc6df187881aaa1b3dc52e71cbe8804b9f9c3225785f1b08ad144dc"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_full_version": "3.9.13",
            "python_version": "3.9"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "anyio": {
            "hashes": [
                "sha256:25ea0d673ae30af41a0c442f81cf3b38c7e79fdc7b60335a4c14e05eb0947421",
                "sha256:fbbe32bd270d2a2ef3ed1c5d45041250284e31fc0a4df4a5a6071842051a51e3"
            ],
            "markers": "python_full_version >= '3.6.2'",
            "version": "==3.6.2"
        },
        "fastapi": {
            "file": "https://files.pythonhosted.org/packages/4e/1a/04887c641b67e6649bde845b9a631f73a7abfbe3afda83957e09b95d88eb/fastapi-0.95.2-py3-none-any.whl",
            "hashes": [
                "sha256:d374dbc4ef2ad9b803899bd3360d34c534adc574546e25314ab72c0c4411749f"
            ],
            "index": "pypi",
            "version": "==0.95.2"
        },
        "idna": {
            "hashes": [
                "sha256:814f528e8dead7d329833b91c5faa87d60bf71824cd12a7530b5526063d02cb4",
                "sha256:90b77e79eaa3eba6de819a0c442c0b4ceefc341a7a2ab77d7562bf49f425c5c2"
            ],
            "markers": "python_version >= '3.5'",
            "version": "==3.4"
        },
        "pydantic": {
            "hashes": [
                "sha256:01aea3a42c13f2602b7ecbbea484a98169fb568ebd9e247593ea05f01b884b2e",
                "sha256:0cd181f1d0b1d00e2b705f1bf1ac7799a2d938cce3376b8007df62b29be3c2c6",
                "sha256:10a86d8c8db68086f1e30a530f7d5f83eb0685e632e411dbbcf2d5c0150e8dcd",
                "sha256:193924c563fae6ddcb71d3f06fa153866423ac1b793a47936656e806b64e24ca",
                "sha256:464855a7ff7f2cc2cf537ecc421291b9132aa9c79aef44e917ad711b4a93163b",
                "sha256:516f1ed9bc2406a0467dd777afc636c7091d71f214d5e413d64fef45174cfc7a",
                "sha256:6434b49c0b03a51021ade5c4daa7d70c98f7a79e95b551201fff682fc1661245",
                "sha256:64d34ab766fa056df49013bb6e79921a0265204c071984e75a09cbceacbbdd5d",
                "sha256:670bb4683ad1e48b0ecb06f0cfe2178dcf74ff27921cdf1606e527d2617a81ee",
                "sha256:68792151e174a4aa9e9fc1b4e653e65a354a2fa0fed169f7b3d09902ad2cb6f1",
                "sha256:701daea9ffe9d26f97b52f1d157e0d4121644f0fcf80b443248434958fd03dc3",
                "sha256:7d45fc99d64af9aaf7e308054a0067fdcd87ffe974f2442312372dfa66e1001d",
                "sha256:80b1fab4deb08a8292d15e43a6edccdffa5377a36a4597bb545b93e79c5ff0a5",
                "sha256:82dffb306dd20bd5268fd6379bc4bfe75242a9c2b79fec58e1041fbbdb1f7914",
                "sha256:8c7f51861d73e8b9ddcb9916ae7ac39fb52761d9ea0df41128e81e2ba42886cd",
                "sha256:950ce33857841f9a337ce07ddf46bc84e1c4946d2a3bba18f8280297157a3fd1",
                "sha256:976cae77ba6a49d80f461fd8bba183ff7ba79f44aa5cfa82f1346b5626542f8e",
                "sha256:9f6f0fd68d73257ad6685419478c5aece46432f4bdd8d32c7345f1986496171e",
                "sha256:a7cd2251439988b413cb0a985c4ed82b6c6aac382dbaff53ae03c4b23a70e80a",
                "sha256:abfb7d4a7cd5cc4e1d1887c43503a7c5dd608eadf8bc615413fc498d3e4645cd",
                "sha256:ae150a63564929c675d7f2303008d88426a0add46efd76c3fc797cd71cb1b46f",
                "sha256:b0f85904f73161817b80781cc150f8b906d521fa11e3cdabae19a581c3606209",
                "sha256:b4a849d10f211389502059c33332e91327bc154acc1845f375a99eca3afa802d",
                "sha256:c15582f9055fbc1bfe50266a19771bbbef33dd28c45e78afbe1996fd70966c2a",
                "sha256:c230c0d8a322276d6e7b88c3f7ce885f9ed16e0910354510e0bae84d54991143",
                "sha256:cc1dde4e50a5fc1336ee0581c1612215bc64ed6d28d2c7c6f25d2fe3e7c3e918",
                "sha256:cf135c46099ff3f919d2150a948ce94b9ce545598ef2c6c7bf55dca98a304b52",
                "sha256:cfc83c0678b6ba51b0532bea66860617c4cd4251ecf76e9846fa5a9f3454e97e",
                "sha256:d2a5ebb48958754d386195fe9e9c5106f11275867051bf017a8059410e9abf1f",
                "sha256:d71e69699498b020ea198468e2480a2f1e7433e32a3a99760058c6520e2bea7e",
                "sha256:d75ae19d2a3dbb146b6f324031c24f8a3f52ff5d6a9f22f0683694b3afcb16fb",
                "sha256:dfe2507b8ef209da71b6fb5f4e597b50c5a34b78d7e857c4f8f3115effaef5fe",
                "sha256:e0cfe895a504c060e5d36b287ee696e2fdad02d89e0d895f83037245218a87fe",
                "sha256:e79e999e539872e903767c417c897e729e015872040e56b96e67968c3b918b2d",
                "sha256:ecbbc51391248116c0a055899e6c3e7ffbb11fb5e2a4cd6f2d0b93272118a209",
                "sha256:f4a2b50e2b03d5776e7f21af73e2070e1b5c0d0df255a827e7c632962f8315af"
            ],
            "markers": "python_version >= '3.7'",
            "version": "==1.10.7"
        },
        "sniffio": {
            "hashes": [
                "sha256:e60305c5e5d314f5389259b7f22aaa33d8f7dee49763119234af3755c55b9101",
                "sha256:eecefdce1e5bbfb7ad2eeaabf7c1eeb404d7757c379bd1f7e5cce9d8bf425384"
            ],
            "markers": "python_version >= '3.7'",
            "version": "==1.3.0"
        },
        "starlette": {
            "hashes": [
                "sha256:6a6b0d042acb8d469a01eba54e9cda6cbd24ac602c4cd016723117d6a7e73b75",
                "sha256:918416370e846586541235ccd38a474c08b80443ed31c578a418e2209b3eef91"
            ],
            "markers": "python_version >= '3.7'",
            "version": "==0.27.0"
        },
        "typing-extensions": {
            "hashes": [
                "sha256:5cb5f4a79139d699607b3ef622a1dedafa84e115ab0024e0d9c044a9479ca7cb",
                "sha256:fb33085c39dd998ac16d1431ebc293a8b3eedd00fd4a32de0ff79002c19511b4"
            ],
            "markers": "python_version >= '3.7'",
            "version": "==4.5.0"
        }
    },
    "develop": {}
}

pipenv requirements output:

$ pipenv requirements
-i https://pypi.org/simple
anyio==3.6.2 ; python_full_version >= '3.6.2'
https://files.pythonhosted.org/packages/4e/1a/04887c641b67e6649bde845b9a631f73a7abfbe3afda83957e09b95d88eb/fastapi-0.95.2-py3-none-any.whl#egg=fastapi
idna==3.4 ; python_version >= '3.5'
pydantic==1.10.7 ; python_version >= '3.7'
sniffio==1.3.0 ; python_version >= '3.7'
starlette==0.27.0 ; python_version >= '3.7'
typing-extensions==4.5.0 ; python_version >= '3.7'

pipenv --version output:

$ pipenv --version
pipenv, version 2023.4.29

[matteius]

Yup looks like this is an issue @petereon and since requirementslib is undergoing a recent massive refactor that is still under review I just checked it on that branch and it still has this issue #5679 -- so I will confirm this as an ongoing bug -- it seems the extras name is captured in the Pipfile but not applied during the resolve phase. We should fix it but in the refactor'd code since that is the path forward.

[petereon]

@matteius, thanks for taking a look. I currently have a workaround with pulling the .whl file locally and installing from the local folder which seems to work.

[matteius]

I am looking for help testing a rather major overhaul that also should address this issue #5818