Pipenv selects wrong PyPi repository for private packages with non canonical names #6056

mangin · 2024-01-05T13:34:43Z

Issue description

Let's assume that we have a package 'some-cool-package-name'. Pip allows non-canonical names using underscores. In this particular case it would look like 'some_cool_package_name`

if the package is located in private PyPi index than pipenv would select default index during creating a lock. That would create problems with locking.

The reason of the problem

In this file we normalize the names of packages:
https://github.com/pypa/pipenv/blob/main/pipenv/patched/pip/_internal/resolution/resolvelib/factory.py#L250

And we pass normalized name to finder:
https://github.com/pypa/pipenv/blob/main/pipenv/patched/pip/_internal/resolution/resolvelib/factory.py#L284

Here we get the repository of packages by package name:

pipenv/pipenv/patched/pip/_internal/models/search_scope.py

Line 142 in 8e06085

index_urls = [self.index_lookup[project_name]]

Here you can see how we create mapping between package name and repository:

pipenv/pipenv/utils/resolver.py

Line 319 in 8e06085

index_mapping[source["name"]] = source["url"]

But there we create mapping between package name and repository. (Pay attention that here we don't use normalized package name)

Expected result

Pipenv should be able to work with packages that have non-canonical names.

Actual result

Pipenv has a problem with locking non-canonical names

Steps to replicate

pipenv install -d

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[[source]]
url = "https://test.pypi.org/simple"
verify_ssl = true
name = "testpypi"

[packages]
pipenv_test_private_package = {version = "*", index = "testpypi"}

Stack trace:

Pipfile.lock not found, creating...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✘ Locking Failed!
⠸ Locking...False
CRITICAL:pipenv.patched.pip._internal.resolution.resolvelib.factory:Could not find a version that satisfies the requirement pipenv_test_private_package (from versions: none)
[ResolutionFailure]:   File "~/.venv/lib/python3.10/site-packages/pipenv/resolver.py", line 645, in _main
[ResolutionFailure]:       resolve_packages(
[ResolutionFailure]:   File "~/.venv/lib/python3.10/site-packages/pipenv/resolver.py", line 612, in resolve_packages
[ResolutionFailure]:       results, resolver = resolve(
[ResolutionFailure]:   File "~/.venv/lib/python3.10/site-packages/pipenv/resolver.py", line 592, in resolve
[ResolutionFailure]:       return resolve_deps(
[ResolutionFailure]:   File "~/.venv/lib/python3.10/site-packages/pipenv/utils/resolver.py", line 908, in resolve_deps
[ResolutionFailure]:       results, hashes, internal_resolver = actually_resolve_deps(
[ResolutionFailure]:   File "~/.venv/lib/python3.10/site-packages/pipenv/utils/resolver.py", line 681, in actually_resolve_deps
[ResolutionFailure]:       resolver.resolve()
[ResolutionFailure]:   File "~/.venv/lib/python3.10/site-packages/pipenv/utils/resolver.py", line 442, in resolve
[ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv run pip install <requirement_name> to bypass this mechanism, then run $ pipenv graph to inspect the versions actually installed in the virtualenv.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: No matching distribution found for pipenv_test_private_package

Traceback (most recent call last):
  File "~/.venv/bin/pipenv", line 8, in <module>
    sys.exit(cli())
  File "~/.venv/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "~/.venv/lib/python3.10/site-packages/pipenv/cli/options.py", line 58, in main
    return super().main(*args, **kwargs, windows_expand_args=False)
  File "~/.venv/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "~/.venv/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "~/.venv/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "~/.venv/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "~/.venv/lib/python3.10/site-packages/pipenv/vendor/click/decorators.py", line 92, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "~/.venv/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "~/.venv/lib/python3.10/site-packages/pipenv/cli/command.py", line 209, in install
    do_install(
  File "~/.venv/lib/python3.10/site-packages/pipenv/routines/install.py", line 164, in do_install
    do_init(
  File "~/.venv/lib/python3.10/site-packages/pipenv/routines/install.py", line 672, in do_init
    do_lock(
  File "~/.venv/lib/python3.10/site-packages/pipenv/routines/lock.py", line 65, in do_lock
    venv_resolve_deps(
  File "~/.venv/lib/python3.10/site-packages/pipenv/utils/resolver.py", line 849, in venv_resolve_deps
    c = resolve(cmd, st, project=project)
  File "~/.venv/lib/python3.10/site-packages/pipenv/utils/resolver.py", line 718, in resolve
    raise RuntimeError("Failed to lock Pipfile.lock!")
RuntimeError: Failed to lock Pipfile.lock!

$ pipenv --support

Pipenv version: '2023.11.15'

Pipenv location: '/.venv/lib/python3.10/site-packages/pipenv'

Python location: '/.venv/bin/python3.10'

OS Name: 'posix'

User pip version: '23.3.1'

user Python installations found:

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.10.4',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.15.0-91-generic',
 'platform_system': 'Linux',
 'platform_version': '#101~20.04.1-Ubuntu SMP Thu Nov 16 14:22:28 UTC 2023',
 'python_full_version': '3.10.4',
 'python_version': '3.10',
 'sys_platform': 'linux'}

System environment variables:

SHELL
SESSION_MANAGER
QT_ACCESSIBILITY
PIPENV_VENV_IN_PROJECT
COLORTERM
XDG_CONFIG_DIRS
NVM_INC
XDG_MENU_PREFIX
GNOME_DESKTOP_SESSION_ID
LC_ADDRESS
GNOME_SHELL_SESSION_MODE
LC_NAME
SSH_AUTH_SOCK
ADFS_USER
XMODIFIERS
DESKTOP_SESSION
LC_MONETARY
SSH_AGENT_PID
PTC_ARTIFACTORY_TOKEN
EDITOR
GTK_MODULES
SYS_DST_APPROVER_STASH_TOKEN
PWD
AD_USERNAME
LOGNAME
XDG_SESSION_DESKTOP
XDG_SESSION_TYPE
GPG_AGENT_INFO
SYSTEM_USER
XAUTHORITY
GJS_DEBUG_TOPICS
WINDOWPATH
SNYK_TOKEN
HOME
USERNAME
IM_CONFIG_PHASE
LC_PAPER
LANG
LS_COLORS
XDG_CURRENT_DESKTOP
VTE_VERSION
ARTIFACTORY_PASSWORD
GNOME_TERMINAL_SCREEN
MARVIN_JENKINS_PASSWORD
INVOCATION_ID
MANAGERPID
GOROOT
SYS_DST_ROBOT_STASH_TOKEN
GJS_DEBUG_OUTPUT
NVM_DIR
LESSCLOSE
XDG_SESSION_CLASS
MARVIN_JENKINS_USER
TERM
LC_IDENTIFICATION
DST_ARTIFACTORY_TOKEN
LESSOPEN
USER
GNOME_TERMINAL_SERVICE
VISUAL
DISPLAY
SHLVL
NVM_CD_FLAGS
LC_TELEPHONE
QT_IM_MODULE
LC_MEASUREMENT
ARTIFACTORY_USERNAME
XDG_RUNTIME_DIR
ARTIFACTORY_TOKEN
LC_TIME
PTC_STASH_TOKEN
RAPID7_TOKEN
JOURNAL_STREAM
XDG_DATA_DIRS
PATH
GDMSESSION
DBUS_SESSION_BUS_ADDRESS
NVM_BIN
LC_NUMERIC
OLDPWD
GOPATH
_
PIP_DISABLE_PIP_VERSION_CHECK
PYTHONDONTWRITEBYTECODE
PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

PIPENV_VENV_IN_PROJECT: True

Debug–specific environment variables:

PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/go/bin:/home/UNKNOWN/dev/bin:/home/UNKNOWN/.local/bin/:/opt/python/3.7.9/bin/
SHELL: /bin/bash
EDITOR: vim
LANG: en_US.UTF-8
PWD: /

Contents of Pipfile ('/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[[source]]
url = "https://test.pypi.org/simple"
verify_ssl = true
name = "testpypi"

[packages]
pipenv_test_private_package = {version = "*", index = "testpypi"}

The text was updated successfully, but these errors were encountered:

Sintayew4 · 2024-03-26T20:57:28Z

#6056

mangin mentioned this issue Jan 5, 2024

Fixed a bug with locking packages with non canonical names #6057

Merged

matteius closed this as completed Jan 27, 2024

matteius added the Type: Regression This issue is a regression of a previous behavior. label Jan 27, 2024

Kroppeb mentioned this issue Mar 26, 2024

Updating packages with non canonical names errors out #6116

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pipenv selects wrong PyPi repository for private packages with non canonical names #6056

Pipenv selects wrong PyPi repository for private packages with non canonical names #6056

mangin commented Jan 5, 2024 •

edited

Loading

Sintayew4 commented Mar 26, 2024

Pipenv selects wrong PyPi repository for private packages with non canonical names #6056

Pipenv selects wrong PyPi repository for private packages with non canonical names #6056

Comments

mangin commented Jan 5, 2024 • edited Loading

Issue description

The reason of the problem

Expected result

Actual result

Steps to replicate

Sintayew4 commented Mar 26, 2024

mangin commented Jan 5, 2024 •

edited

Loading