Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modularization of sagelib: Distributions sagemath-{brial,combinat,eclib,flint,gap,giac,glpk,graphs,groups,homfly,lcalc,libbraiding,libecm,linbox,modules,mpmath,ntl,pari,plot,polyhedra,schemes,singular,standard-no-symbolics,symbolics} #35095

Draft
wants to merge 4,103 commits into
base: develop
Choose a base branch
from

Conversation

mkoeppe
Copy link
Member

@mkoeppe mkoeppe commented Feb 12, 2023

📚 Description

Screenshot 2023-09-09 at 11 57 02 PM

📝 Checklist

  • I have made sure that the title is self-explanatory and the description concisely explains the PR.
  • I have linked an issue or discussion.
  • I have created tests covering the changes.
  • I have updated the documentation accordingly.

⌛ Dependencies

This is part of:

@mkoeppe mkoeppe changed the title Modularization of sagelib: Break out a separate package **sagemath-polyhedra** Modularization of sagelib: Break out a separate package sagemath-polyhedra Feb 12, 2023
@mkoeppe mkoeppe added this to the sage-9.9 milestone Feb 12, 2023
@codecov-commenter
Copy link

codecov-commenter commented Feb 14, 2023

Codecov Report

Patch coverage: 63.80% and project coverage change: -1.22 ⚠️

Comparison is base (c00e6c2) 88.62% compared to head (ef82aaa) 87.40%.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop   #35095      +/-   ##
===========================================
- Coverage    88.62%   87.40%   -1.22%     
===========================================
  Files         2148     2180      +32     
  Lines       398855   399144     +289     
===========================================
- Hits        353480   348878    -4602     
- Misses       45375    50266    +4891     
Impacted Files Coverage Δ
src/sage/all__sagemath_categories.py 0.00% <0.00%> (ø)
src/sage/all__sagemath_polyhedra.py 0.00% <0.00%> (ø)
src/sage/arith/all.py 100.00% <ø> (ø)
src/sage/arith/misc.py 90.49% <ø> (-0.50%) ⬇️
src/sage/calculus/test_sympy.py 0.00% <ø> (ø)
src/sage/categories/additive_magmas.py 97.77% <ø> (ø)
src/sage/categories/additive_semigroups.py 100.00% <ø> (ø)
src/sage/categories/affine_weyl_groups.py 100.00% <ø> (ø)
src/sage/categories/algebra_functor.py 80.00% <ø> (-20.00%) ⬇️
src/sage/categories/algebras.py 100.00% <ø> (ø)
... and 232 more

... and 283 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@mkoeppe mkoeppe force-pushed the t/32432/modularization_of_sagelib__break_out_a_separate_package_sagemath_polyhedra branch 2 times, most recently from ee6eb49 to d368b6e Compare February 17, 2023 19:49
@mkoeppe mkoeppe self-assigned this Feb 18, 2023
@kwankyu
Copy link
Collaborator

kwankyu commented Feb 22, 2023

I ran

./bootstrap && ./sage -sh -c '(cd pkgs/sagemath-polyhedra && SAGE_NUM_THREADS=16 tox -v -v -v)'

after full rebuild on macOS monterey, and got

            [37/54] gcc -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I./sage/cpython -I/private/var/folders/td/fw1q9ljs311ggyph77rs53_40000gn/T/pip-build-env-_on11jf5/overlay/lib/python3.11/site-packages/gmpy2 -I./sage/rings -I/private/var/folders/td/fw1q9ljs311ggyph77rs53_40000gn/T/pip-build-env-_on11jf5/overlay/lib/python3.11/site-packages -I/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.11/include/python3.11 -Ibuild/cythonized -I/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-polyhedra/.tox/.pkg/include -I/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.11/include/python3.11 -c build/cythonized/sage/structure/coerce.c -o build/temp.macosx-12-x86_64-cpython-311/build/cythonized/sage/structure/coerce.o -fno-strict-aliasing -DCYTHON_CLINE_IN_TRACEBACK=1
            build/cythonized/sage/structure/coerce.c:779:10: fatal error: 'gmp.h' file not found
            #include "gmp.h"
                     ^~~~~~~
            1 error generated.

@mkoeppe
Copy link
Member Author

mkoeppe commented Feb 22, 2023

I've updated the instructions for testing.

@kwankyu
Copy link
Collaborator

kwankyu commented Feb 23, 2023

sagemath-polyhedra fails to build for me. It says gmp.h is missing.

@mkoeppe
Copy link
Member Author

mkoeppe commented Feb 23, 2023

Is this on macOS with homebrew? DId you run .homebrew-build-env?

@kwankyu
Copy link
Collaborator

kwankyu commented Feb 23, 2023

Yes. My bash profile does source /Users/kwankyu/GitHub/sage-dev/.homebrew-build-env.

I manually pasted the line and tried again. But I got the same.

@kwankyu
Copy link
Collaborator

kwankyu commented Feb 23, 2023

I am trying again after deleting pkgs/sagemath-polyhedra/.tox directory.

@kwankyu
Copy link
Collaborator

kwankyu commented Feb 23, 2023

I got the same failure. Part of the log

.pkg: 89422 W install_requires> python -I -m pip install 'cysignals>=1.10.2' 'cython<1.0,>=0.29.21' 'gmpy2>=2.1.0' memory_allocator 'pkgconfig>=1.5.1' 'sage-setup~=10.0b1' 'sagemath-categories~=10.0b1' 'sagemath-environment~=10.0b1' 'setuptools>=49.6.0' 'wheel>=0.36.2' [tox/tox_env/api.py:428]
Collecting cysignals>=1.10.2
  Using cached cysignals-1.11.2-cp311-cp311-macosx_12_0_x86_64.whl
Collecting cython<1.0,>=0.29.21
  Using cached Cython-0.29.33-py2.py3-none-any.whl (987 kB)
Collecting gmpy2>=2.1.0
  Using cached gmpy2-2.1.5-cp311-cp311-macosx_10_9_x86_64.whl (806 kB)
Collecting memory_allocator
  Using cached memory_allocator-0.1.3.tar.gz (19 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting pkgconfig>=1.5.1
  Using cached pkgconfig-1.5.5-py3-none-any.whl (6.7 kB)
Collecting sage-setup~=10.0b1
  Using cached sage_setup-10.0b1-py3-none-any.whl
Collecting sagemath-categories~=10.0b1
  Using cached sagemath-categories-10.0b1.tar.gz (504 kB)
  Installing build dependencies ... error
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [946 lines of output]

@mkoeppe
Copy link
Member Author

mkoeppe commented Feb 23, 2023

Have you tried the command make wheels?

@kwankyu
Copy link
Collaborator

kwankyu commented Feb 23, 2023

No. Shall I?

@mkoeppe
Copy link
Member Author

mkoeppe commented Feb 23, 2023

Yes, that's what I would recommend as the best way of testing

@kwankyu
Copy link
Collaborator

kwankyu commented Feb 23, 2023

I ran make wheels. It ended with

[sagemath_objects-10.0.beta1] * Creating venv isolated environment...
[sagemath_objects-10.0.beta1] * Installing packages in isolated environment... (cysignals >=1.10.2, cython >=0.29.21, <1.0, gmpy2 >=2.1.0, sage-setup ~= 10.0b1, sagemath-environment ~= 10.0b1, setuptools >=49.6.0, wheel >=0.36.2)
[sagemath_objects-10.0.beta1] Looking in links: file:///Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/var/lib/sage/wheels
[sagemath_objects-10.0.beta1] Processing /Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/var/lib/sage/wheels/sagemath_environment-10.0b1-py3-none-any.whl
[sagemath_objects-10.0.beta1] Processing /Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/var/lib/sage/wheels/setuptools-65.6.3-py3-none-any.whl
[sagemath_objects-10.0.beta1] Processing /Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/var/lib/sage/wheels/Cython-0.29.32-cp311-cp311-macosx_12_0_x86_64.whl
[sagemath_objects-10.0.beta1] Processing /Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/var/lib/sage/wheels/gmpy2-2.1.2-cp311-cp311-macosx_12_0_x86_64.whl
[sagemath_objects-10.0.beta1] ERROR: Could not find a version that satisfies the requirement sage-setup~=10.0b1 (from versions: none)
[sagemath_objects-10.0.beta1] ERROR: No matching distribution found for sage-setup~=10.0b1
[sagemath_objects-10.0.beta1] 
[sagemath_objects-10.0.beta1] Traceback (most recent call last):
[sagemath_objects-10.0.beta1]   File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/lib/python3.11/site-packages/build/__main__.py", line 375, in main
[sagemath_objects-10.0.beta1]     built = build_call(
[sagemath_objects-10.0.beta1]             ^^^^^^^^^^^
[sagemath_objects-10.0.beta1]   File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/lib/python3.11/site-packages/build/__main__.py", line 235, in build_package_via_sdist
[sagemath_objects-10.0.beta1]     sdist = _build(isolation, builder, outdir, 'sdist', config_settings, skip_dependency_check)
[sagemath_objects-10.0.beta1]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[sagemath_objects-10.0.beta1]   File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/lib/python3.11/site-packages/build/__main__.py", line 145, in _build
[sagemath_objects-10.0.beta1]     return _build_in_isolated_env(builder, outdir, distribution, config_settings)
[sagemath_objects-10.0.beta1]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[sagemath_objects-10.0.beta1]   File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/lib/python3.11/site-packages/build/__main__.py", line 113, in _build_in_isolated_env
[sagemath_objects-10.0.beta1]     env.install(builder.build_system_requires)
[sagemath_objects-10.0.beta1]   File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/lib/python3.11/site-packages/build/env.py", line 214, in install
[sagemath_objects-10.0.beta1]     _subprocess(cmd)
[sagemath_objects-10.0.beta1]   File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/lib/python3.11/site-packages/build/env.py", line 79, in _subprocess
[sagemath_objects-10.0.beta1]     raise e
[sagemath_objects-10.0.beta1]   File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/lib/python3.11/site-packages/build/env.py", line 76, in _subprocess
[sagemath_objects-10.0.beta1]     subprocess.run(cmd, check=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
[sagemath_objects-10.0.beta1]   File "/usr/local/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 571, in run
[sagemath_objects-10.0.beta1]     raise CalledProcessError(retcode, process.args,
[sagemath_objects-10.0.beta1] subprocess.CalledProcessError: Command '['/private/var/folders/td/fw1q9ljs311ggyph77rs53_40000gn/T/build-env-y415lely/bin/python', '-Im', 'pip', 'install', '--use-pep517', '--no-warn-script-location', '-r', '/var/folders/td/fw1q9ljs311ggyph77rs53_40000gn/T/build-reqs-t6wqxz1a.txt']' returned non-zero exit status 1.
[sagemath_objects-10.0.beta1] 
[sagemath_objects-10.0.beta1] ERROR Command '['/private/var/folders/td/fw1q9ljs311ggyph77rs53_40000gn/T/build-env-y415lely/bin/python', '-Im', 'pip', 'install', '--use-pep517', '--no-warn-script-location', '-r', '/var/folders/td/fw1q9ljs311ggyph77rs53_40000gn/T/build-reqs-t6wqxz1a.txt']' returned non-zero exit status 1.
[sagemath_objects-10.0.beta1] ********************************************************************************
[sagemath_objects-10.0.beta1] Failure building sdist and wheel
[sagemath_objects-10.0.beta1] ********************************************************************************
make[3]: *** [sagemath_objects-SAGE_VENV-no-deps] Error 1
make[2]: *** [/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/var/lib/sage/installed/sagemath_objects-10.0.beta1] Error 2
make[1]: *** [wheels] Error 2

real	0m39.310s
user	0m29.301s
sys	0m8.784s
***************************************************************
Error building Sage.

The following package(s) may have failed to build (not necessarily
during this run of 'make wheels'):

It is safe to delete any log files and build directories, but they
contain information that is helpful for debugging build problems.
WARNING: If you now run 'make' again, the build directory of the
same version of the package will, by default, be deleted. Set the
environment variable SAGE_KEEP_BUILT_SPKGS=yes to prevent this.

make: *** [wheels] Error 1

@mkoeppe
Copy link
Member Author

mkoeppe commented Feb 23, 2023

Could you try if ./bootstrap fixes this problem?

@kwankyu
Copy link
Collaborator

kwankyu commented Feb 23, 2023

I ran ./bootstrap && make wheels. I got the same error.

@mkoeppe
Copy link
Member Author

mkoeppe commented Feb 23, 2023

Could you check the contents of venv/var/lib/sage/wheels/sage*.whl please?

@kwankyu
Copy link
Collaborator

kwankyu commented Feb 23, 2023

$ ls venv/var/lib/sage/wheels/sage*.whl
venv/var/lib/sage/wheels/sage_sws2rst-10.0b1-py3-none-any.whl
venv/var/lib/sage/wheels/sagemath_environment-10.0b1-py3-none-any.whl
venv/var/lib/sage/wheels/sagenb_export-3.3-py3-none-any.whl
venv/var/lib/sage/wheels/sagetex-3.6.1-py3-none-any.whl

@mkoeppe
Copy link
Member Author

mkoeppe commented Feb 23, 2023

The sage_setup wheel is missing, which should have been one of the first wheels that make wheels makes.

@kwankyu
Copy link
Collaborator

kwankyu commented Feb 23, 2023

Part of the log:

make --no-print-directory SAGE_EDITABLE=no SAGE_WHEELS=yes sage_sws2rst sage_setup sagemath_environment sagemath_objects sagemath_repl sagemath_categories sagemath_polyhedra sage_conf sagelib sage_docbuild
make[2]: Nothing to be done for `sage_sws2rst'.
make[2]: Nothing to be done for `sage_setup'.
make --no-print-directory python_build-no-deps
sage-logger -p 'sage --pip install -r "/Users/kwankyu/GitHub/sage-dev/build/pkgs/python_build/requirements.txt"' '/Users/kwankyu/GitHub/sage-dev/logs/pkgs/python_build.log'
[python_build] Requirement already satisfied: build in /Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/lib/python3.11/site-packages (from -r /Users/kwankyu/GitHub/sage-dev/build/pkgs/python_build/requirements.txt (line 1)) (0.10.0)
[python_build] Requirement already satisfied: packaging>=19.0 in /Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/lib/python3.11/site-packages (from build->-r /Users/kwankyu/GitHub/sage-dev/build/pkgs/python_build/requirements.txt (line 1)) (21.3)
[python_build] Requirement already satisfied: pyproject_hooks in /Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/lib/python3.11/site-packages (from build->-r /Users/kwankyu/GitHub/sage-dev/build/pkgs/python_build/requirements.txt (line 1)) (1.0.0)
[python_build] Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/lib/python3.11/site-packages (from packaging>=19.0->build->-r /Users/kwankyu/GitHub/sage-dev/build/pkgs/python_build/requirements.txt (line 1)) (3.0.9)
make --no-print-directory sagemath_objects-SAGE_VENV-no-deps
[sagemath_objects-10.0.beta1] * Creating venv isolated environment...
[sagemath_objects-10.0.beta1] * Installing packages in isolated environment... (cysignals >=1.10.2, cython >=0.29.21, <1.0, gmpy2 >=2.1.0, sage-setup ~= 10.0b1, sagemath-environment ~= 10.0b1, setuptools >=49.6.0, wheel >=0.36.2)
[sagemath_objects-10.0.beta1] Looking in links: file:///Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/var/lib/sage/wheels
[sagemath_objects-10.0.beta1] Processing /Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/var/lib/sage/wheels/wheel-0.38.4-py3-none-any.whl
[sagemath_objects-10.0.beta1] Processing /Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/var/lib/sage/wheels/sagemath_environment-10.0b1-py3-none-any.whl
[sagemath_objects-10.0.beta1] Processing /Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/var/lib/sage/wheels/gmpy2-2.1.2-cp311-cp311-macosx_12_0_x86_64.whl
[sagemath_objects-10.0.beta1] Processing /Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/var/lib/sage/wheels/cysignals-1.11.2-cp311-cp311-macosx_12_0_x86_64.whl
[sagemath_objects-10.0.beta1] Processing /Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11/var/lib/sage/wheels/setuptools-65.6.3-py3-none-any.whl
[sagemath_objects-10.0.beta1] ERROR: Could not find a version that satisfies the requirement sage-setup~=10.0b1 (from versions: none)
[sagemath_objects-10.0.beta1] ERROR: No matching distribution found for sage-setup~=10.0b1
[sagemath_objects-10.0.beta1] 

@kwankyu
Copy link
Collaborator

kwankyu commented Feb 23, 2023

It says Nothing to be done for sage_setup!

…ect] dependencies, [project.optional-dependencies] too
…package names, treat dashes and underscores as equivalent
…b__break_out_a_separate_package_sagemath_polyhedra
vbraun pushed a commit to vbraun/sage that referenced this pull request Jul 20, 2024
…eds`

    
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->

Also some doctest cosmetics.

- Cherry-picked from sagemath#35095.

`sage --fixdoctests --verbose` now shows doctest tagging statistics.

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [x] The title is concise and informative.
- [ ] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#38118
Reported by: Matthias Köppe
Reviewer(s): Kwankyu Lee, Matthias Köppe
vbraun pushed a commit to vbraun/sage that referenced this pull request Jul 20, 2024
    
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->

- Cherry-picked from sagemath#35095.

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [x] The title is concise and informative.
- [ ] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#38142
Reported by: Matthias Köppe
Reviewer(s): Kwankyu Lee, Matthias Köppe
vbraun pushed a commit to vbraun/sage that referenced this pull request Jul 20, 2024
    
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->

Some of these are from the use of a matrix norm, which `# needs scipy`.

Also a norm-related change in `sage.modules.free_module_integer` -
avoiding use of `norm`, which send us into symbolics land, in favor of
just comparing norm squares

- Cherry-picked from sagemath#35095.

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [x] The title is concise and informative.
- [x] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#38236
Reported by: Matthias Köppe
Reviewer(s): David Coudert
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment