Poetry removes numpy for some versions of python and pandas

[sam-hoffman]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name:

System
Platform: darwin
OS:       posix
Python:   /usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.10

• Poetry version: Poetry version 1.1.12

Issue

pyproject.toml:

[tool.poetry]
name = "test"
version = "0.1.0"
description = ""
authors = [""]

[tool.poetry.dependencies]
python = "^3.9"
pandas = "^1.3.0"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

If I run poetry update with the pyproject.toml shown above, the correct dependencies are installed. If I change the Python version to python = "~3.9" and run poetry update again, poetry shows:

Updating dependencies
Resolving dependencies... (0.1s)

Writing lock file

Package operations: 0 installs, 0 updates, 1 removal

  • Removing numpy (1.22.1)

This seems wrong, since pandas won't run without numpy. What's going on here?

[radoering]

I can't reproduce the issue completely, so I can only guess what's going on. I suppose you are using python 3.10. When running poetry update with the initial pyproject.toml, a virtualenv with python 3.10 is created and everything is fine. When looking into poetry.lock you should see the following lines below the pandas package:

[package.dependencies]
numpy = [
    {version = ">=1.18.5", markers = "platform_machine != \"aarch64\" and platform_machine != \"arm64\" and python_version < \"3.10\""},
    {version = ">=1.19.2", markers = "platform_machine == \"aarch64\" and python_version < \"3.10\""},
    {version = ">=1.20.0", markers = "platform_machine == \"arm64\" and python_version < \"3.10\""},
    {version = ">=1.21.0", markers = "python_version >= \"3.10\""},
]

When changing the python constraint to ~3.9, python 3.10 is not supported anymore by your project. When running poetry update again, the dependencies are locked again and the following line is removed from poetry.lock because python >= 3.10 is not supported anymore:

    {version = ">=1.21.0", markers = "python_version >= \"3.10\""},

Unfortunately, poetry does not complain about the virtualenv using python 3.10, which is not supported anymore. Considering the missing line, it seems that pandas does not depend on numpy for python 3.10. Thus, the dependency is removed. Actually, I think poetry should give an error about the unsupported python version.

When changing the python constraint to python = "~3.9". you probably want to have a virtualenv using python 3.9 and not python 3.10. In order to achieve this, you should run poetry env use python3.9 (or similar) before running poetry update.

[sam-hoffman]

Two questions:

1. Does Poetry not set the Python version as part of the environment building/loading process? I'm used to Conda where I could just set the version of Python I wanted in the environment.yml and Conda would handle ensuring that I was on the correct version of Python, downloading it if necessary
2. Numpy is definitely required for all versions of Pandas (trying to load Pandas without Numpy installed raises an error). This seems like it might be a Poetry bug?

[radoering]

1. Does Poetry not set the Python version as part of the environment building/loading process?

Depending on the python constraint in your pyproject.toml, several python versions can be used to build the environment (you can even build several environments). You can tell poetry which one to use by calling poetry env command (see documentation for details). If you don't tell poetry which version to use it "will try to use the currently activated Python version". If this version does not comply with pyproject.toml an error is raised.

2. Numpy is definitely required for all versions of Pandas (trying to load Pandas without Numpy installed raises an error). This seems like it might be a Poetry bug?

At least, it's an unsupported workflow. If you start from scratch without an already existing environment with python = "~3.9" in your pyproject.toml, you will get an error if you try to use python 3.10 for building the environment. In your workflow, the issue is that an environment for python 3.10 has already been created (when it was valid to use) and is the current active environment when changing the python constraint in the pyproject.toml.

I think poetry should at least raise an error instead of using the existing enviroment and doing weird things like uninstalling numpy. The current behavior can be considered a bug.

[radoering]

Probably related to #3426. Thus, may be fixed by #4520.