Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pip dependency resolution conflict with requirements.txt #63

Open
rossbar opened this issue Sep 10, 2021 · 10 comments
Open

Pip dependency resolution conflict with requirements.txt #63

rossbar opened this issue Sep 10, 2021 · 10 comments

Comments

@rossbar
Copy link
Contributor

rossbar commented Sep 10, 2021

The first step of make html is pip install -q -r requirements.txt, but unfortunately it is not currently possible to install the dependencies with pip due to conflicts in pip's dependency resolver. The failure mode is particularly frustrating because it sends pip down a path where it tries to download very old versions of e.g. pytest and will only fail after these fail to build.

I've encountered this problem before with the executablebooks project (executablebooks/MyST-NB#333, executablebooks/MyST-NB#289) so I suspected jupyter-book/jupytext as the culprit. Indeed, if jupyterbook and jupytext are removed from the requirements file and installed separately, the failure is resolved.

@hmaarrfk
Copy link
Member

I think this is a pretty big challenge.

It seems to me that sphinx 4.0 has been quite controversial.

  • jupyter-book doesn't want to use it.
  • furo needs it

@hmaarrfk
Copy link
Member

I'm trying to run things with:

diff --git a/Makefile b/Makefile
index 30b70b8..701f261 100644
--- a/Makefile
+++ b/Makefile
@@ -6,7 +6,7 @@ LESSONS_DIR = lessons
 GENERATED_LESSONS_DIR = book/lessons
 
 _requirements.installed:
-       pip install -q -r requirements.txt
+       pip install -r requirements.txt
        touch _requirements.installed
 
 MARKDOWNS = $(wildcard $(LESSONS_DIR)/*.md)
diff --git a/requirements.txt b/requirements.txt
index ea8af47..e7b464c 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,15 +1,12 @@
-scikit-image[data] >= 0.18
-numpy >= 1.12
-scipy >= 1.0
-matplotlib >= 2.1
-notebook >= 4.0
-scikit-learn >= 0.18
-jupyter-book >= 0.10.2
-napari[all]
+scikit-image[data] >= 0.18.3
+matplotlib >=3.4.3
+notebook >=6.4.3
+scikit-learn >=0.24.2
+jupyter-book >=0.11.3
+napari[all] >=0.4.11
 jupytext >=1.10.3
 sphinx_autodoc_typehints>=1.11.0
-ghp-import
-pytest
-pytest-qt
-pooch
-furo
\ No newline at end of file
+ghp-import >=2.0.1
+pytest >=6.0.1
+pytest-qt >=4.0.1
+pooch >=1.5.1

but i get an error called:

touch _requirements.installed
make[1]: *** book: No such file or directory.  Stop.
make: *** [Makefile:37: html] Error 2

@hmaarrfk
Copy link
Member

I presume that this is the focus of
#59

@rossbar
Copy link
Contributor Author

rossbar commented Sep 13, 2021

I presume that this is the focus of #59

To some degree - splitting up the "build" dependencies (e.g. the executablebook project deps) and the dependencies needed to actually run the notebooks can help IME, but it still might require some tweaking to get everything installed.

@hmaarrfk
Copy link
Member

My guess is that something happened in pip to update their "solver" which is causing these headaches.

@rossbar
Copy link
Contributor Author

rossbar commented Sep 13, 2021

Yeah there are multiple potential failure modes. Changes to the pip resolver have caused problems recently, see e.g. pypa/pip#10201, but there have also been dependency conflicts between e.g. jupytext and myst-nb (e.g. executablebooks/MyST-NB#333)

@hmaarrfk
Copy link
Member

is there a solution that includes downgrading pip?

@jni
Copy link
Member

jni commented Sep 13, 2021

@hmaarrfk

I think this is a pretty big challenge.

It seems to me that sphinx 4.0 has been quite controversial.

  • jupyter-book doesn't want to use it.
  • furo needs it

Sorry for the off-topic post, but this is such an ironically relevant story for the scikit-image 1.0 transition! 😂 I'm very glad the discussion settled on skimage2.

@hmaarrfk
Copy link
Member

issue to follow

jupyter-book/jupyter-book#1438

@notatallshaw
Copy link

FYI I tested the requirements.txt with the changes I am proposing here: pypa/pip#10479

And it didn't get stuck backtracking but very quickly gives the error:

ERROR: Cannot install -r .\requirements.txt (line 10), -r .\requirements.txt (line 15) and -r .\requirements.txt (line 7) because these package versions have conflicting dependencies.

The conflict is caused by:
    jupyter-book 0.10.2 depends on sphinx<4 and >=2
    sphinx-autodoc-typehints 1.11.0 depends on Sphinx>=3.0
    furo 2021.9.8 depends on sphinx~=4.0
    jupyter-book 0.10.2 depends on sphinx<4 and >=2
    sphinx-autodoc-typehints 1.11.0 depends on Sphinx>=3.0
    furo 2021.8.31 depends on sphinx~=4.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants