TEST: 1.9.x + blas variants by h-vetinari · Pull Request #223 · conda-forge/scipy-feedstock

h-vetinari · 2023-01-27T08:07:04Z

Following the same scheme as #199, #172, #153 & #130. Should not be merged for the same reasons as #153.

…nda-forge-pinning 2023.01.26.16.59.55

conda-forge-webservices · 2023-01-27T08:07:09Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

h-vetinari · 2023-01-27T10:16:45Z

Scipy 1.9.3

I had gotten stuck in #199 and didn't get around to doing the analysis for the 1.9 series yet. Now picking this up again to have a reasonable chain record of the changes from here to 1.10.

TL;DR: Improvements all around

From 40 failures out of 110 runs for 1.8.1, we're now at 32 failures out of 96 CI runs (added CPython 3.11, no PyPy, distinguished avx512 absence/presence for openblas)

Notable

No PyPy builds (progressing in Re-enable pypy #208)
No Propack tests (progressing in WIP: Re-enable PROPACK tests #200)
Emulation problems on PPC might finally be over?! at least it's looking like that for numpy

The good news:

failures related to TestCOO from 1.8.1 disappeared
lots of other test failures disappeared

The bad news

Recurring problem cases test_x0_equals_Mb[bicgstab] & test_lobpcg.py::test_failure_to_run_iterations
More AVX512-specific failures

Restrictions

aarch64/ppc64le only run tests with label='fast', not the full test suite
the following test skips:

# these multithreading tests occasionally hang in emulation
{% set tests_to_skip = tests_to_skip + " or test_immediate_updating" %}         # [aarch64 or ppc64le]
{% set tests_to_skip = tests_to_skip + " or test_mixed_threads_processes" %}    # [aarch64 or ppc64le]
{% set tests_to_skip = tests_to_skip + " or test_multiprocess[fft]" %}          # [aarch64 or ppc64le]
{% set tests_to_skip = tests_to_skip + " or (test__differential_evolution and test_parallel)" %}  # [aarch64 or ppc64le]
# on the slowest agents, these tests take more than 20min in emulation
{% set tests_to_skip = tests_to_skip + " or (test_lsq_linear and test_large_rank_deficient)" %}   # [aarch64 or ppc64le]
{% set tests_to_skip = tests_to_skip + " or (test_iterative and test_precond_inverse[case1])" %}  # [aarch64 or ppc64le]

Details

lib	for 1.8.1	now	updated version
`scipy`	`1.8.1`	`1.9.3`	X
`numpy`	`1.24.1`	`1.24.1`
`libblas`	`3.9.0-16`	`3.9.0-16`
`blis`	`0.9.0-0`	`0.9.0-0`
`openblas`	`0.3.21-pthreads-3`	`0.3.21-pthreads-3`
`mkl`	`2022.1.0-915`	`2022.1.0-915`
`netlib`	`3.9.0-5`	`3.9.0-5`
`pypy`	`7.3.9-7`	not used
`pythran`	`0.12.1-0`	`0.12.1-0`
`qemu-user-static`	`7.2.0-1`	`7.2.0-1`

variant	1.8.1	1.9.3
all builds	7 test failures related to `TestCOO`	gone 🥳
linux + x64 + mkl + avx512	2 test failures; incl. scipy/scipy#15533	only scipy/scipy#15533
linux + x64 + openblas + non-avx512	✔️	1 test failure scipy/scipy#16927
linux + x64 + netlib	1-2 test failures depending on absence/presence of avx512	passes 🥳
linux + x64 + openblas	2-3 test failures depending on absence/presence of avx512	passes 🥳
linux + aarch64 + openblas	2 test failures	1 test failure
linux + ppc64le	test suite disabled due to emulation problems	cautious signs that this might finally get solved 🥳
osx + accelerate	8 test failures	7 test failures
osx + blis	✔️	1 test failure scipy/scipy#16927
osx + x64 + mkl	2 failures	passes 🥳
osx + x64 + openblas	3 failures	passes 🥳
win + mkl + avx512	1 test failure; scipy/scipy#15533	remains
win + openblas	8-9 test failures depending on absence/presence of avx512	7 test failures

variant	accelerate	blis	mkl w/ avx512	mkl w/o avx512	netlib	openblas w/ avx512	openblas w/o avx512	sum**
linux / x64	➖	✔️	❌	✔️	✔️	✔️	❌	8F
linux / aarch64	➖	➖	➖	➖	✔️	➖	❌	4F
linux / ppc64le	➖	➖	➖	➖	❔	➖	❔	-
osx / arm	❔	➖	➖	➖	❔	➖	❔	-
osx / x64	❌	❌	➖	✔️	✔️	➖	✔️	8F
win / x64	➖	✔️	❌	✔️	✔️	❌	❌	12F
sum**	4F	4F	8F	-	-	4F	12F	32F

** sum of Segfaults (S), Failures (F), resp. Timeouts (T); out of a total of 96 CI combinations being tested (the openblas builds haven't actually been split into avx512 present/absent, but I inferred them from the others)

Build logs:
Azure

linux + mkl + avx512: 1 failure

=========================== short test summary info ============================
FAILED sparse/linalg/_isolve/tests/test_iterative.py::test_x0_equals_Mb[bicgstab] - AssertionError: 
Items are not equal:
 ACTUAL: -11
 DESIRED: 0
= 1 failed, 49590 passed, 2881 skipped, 253 xfailed, 9 xpassed, 477 warnings in 956.56s (0:15:56) =

linux + openblas + non-avx512: 1 failure

=========================== short test summary info ============================
FAILED sparse/linalg/_eigen/lobpcg/tests/test_lobpcg.py::test_failure_to_run_iterations - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
The list of emitted warnings is: [].
= 1 failed, 49589 passed, 2882 skipped, 247 xfailed, 15 xpassed, 481 warnings in 1249.46s (0:20:49) =

linux + aarch64 + openblas: 1 failure

=========================== short test summary info ============================
FAILED linalg/tests/test_solvers.py::test_solve_discrete_are - AssertionError: 
Arrays are not almost equal to 2 decimals

Mismatched elements: 1 / 4 (25%)
Max absolute difference: 0.01635742
Max relative difference: inf
 x: array([[0.00e+00, 1.23e-20],
       [1.23e-20, 1.64e-02]])
 y: array([[0., 0.],
       [0., 0.]])
= 1 failed, 37670 passed, 2848 skipped, 135 xfailed, 12 xpassed, 3231 warnings in 5731.15s (1:35:31) =

windows + mkl + avx512: 1 failure

=========================== short test summary info ===========================
FAILED sparse/linalg/_isolve/tests/test_iterative.py::test_x0_equals_Mb[bicgstab]
= 1 failed, 49097 passed, 3374 skipped, 249 xfailed, 13 xpassed, 488 warnings in 1137.86s (0:18:57) =

windows + openblas: 7 failures

=========================== short test summary info ===========================
FAILED linalg/tests/test_blas.py::TestFBLAS2Simple::test_syr_her - AssertionE...
FAILED linalg/tests/test_decomp.py::TestEig::test_bad_geneig - AssertionError: 
FAILED linalg/tests/test_decomp_ldl.py::test_simple - AssertionError: 
FAILED linalg/tests/test_decomp_ldl.py::test_ldl_type_size_combinations_complex[30-complex64]
FAILED linalg/tests/test_decomp_ldl.py::test_ldl_type_size_combinations_complex[30-complex128]
FAILED linalg/tests/test_decomp_ldl.py::test_ldl_type_size_combinations_complex[150-complex64]
FAILED linalg/tests/test_decomp_ldl.py::test_ldl_type_size_combinations_complex[150-complex128]
= 7 failed, 49092 passed, 3373 skipped, 249 xfailed, 13 xpassed, 478 warnings in 1161.57s (0:19:21) =

osx + accelerate: 7 failures

=========================== short test summary info ============================
FAILED linalg/tests/test_cython_blas.py::TestWfuncPointers::test_float_args
FAILED sparse/linalg/_eigen/tests/test_svds.py::Test_SVDS_LOBPCG::test_svd_v0 - AssertionError: 
Arrays are not equal
item=0

Mismatched elements: 11 / 100 (11%)
Max absolute difference: 2.77555756e-17
Max relative difference: 2.72996743e-16
 x: array([[-0.109858],
       [-0.106207],
       [-0.108685],...
 y: array([[-0.109858],
       [-0.106207],
       [-0.108685],...

FAILED sparse/linalg/_eigen/tests/test_svds.py::Test_SVDS_LOBPCG::test_svd_random_state - AssertionError: 
Arrays are not equal
item=0

Mismatched elements: 43 / 100 (43%)
Max absolute difference: 4.16333634e-17
Max relative difference: 4.02567023e-16
 x: array([[-0.109858],
       [-0.106207],
       [-0.108685],...
 y: array([[-0.109858],
       [-0.106207],
       [-0.108685],...

FAILED sparse/linalg/_eigen/tests/test_svds.py::Test_SVDS_LOBPCG::test_svd_random_state_2[0] - AssertionError: 
Arrays are not equal
item=0

Mismatched elements: 63 / 100 (63%)
Max absolute difference: 4.16333634e-17
Max relative difference: 4.15574571e-16
 x: array([[-0.109858],
       [-0.106207],
       [-0.108685],...
 y: array([[-0.109858],
       [-0.106207],
       [-0.108685],...

FAILED sparse/linalg/_eigen/tests/test_svds.py::Test_SVDS_LOBPCG::test_svd_random_state_2[1] - AssertionError: 
Arrays are not equal
item=0

Mismatched elements: 50 / 100 (50%)
Max absolute difference: 2.77555756e-17
Max relative difference: 3.40554184e-16
 x: array([[-0.109858],
       [-0.106207],
       [-0.108685],...
 y: array([[-0.109858],
       [-0.106207],
       [-0.108685],...

FAILED sparse/linalg/_eigen/tests/test_svds.py::Test_SVDS_LOBPCG::test_svd_random_state_2[random_state2] - AssertionError: 
Arrays are not equal
item=0

Mismatched elements: 77 / 100 (77%)
Max absolute difference: 2.22044605e-16
Max relative difference: 2.02120457e-15
 x: array([[-0.109858],
       [-0.106207],
       [-0.108685],...
 y: array([[-0.109858],
       [-0.106207],
       [-0.108685],...

FAILED sparse/linalg/_eigen/tests/test_svds.py::Test_SVDS_LOBPCG::test_svd_random_state_2[random_state3] - AssertionError: 
Arrays are not equal
item=0

Mismatched elements: 68 / 100 (68%)
Max absolute difference: 2.22044605e-16
Max relative difference: 2.02120457e-15
 x: array([[-0.109858],
       [-0.106207],
       [-0.108685],...
 y: array([[-0.109858],
       [-0.106207],
       [-0.108685],...
= 7 failed, 49573 passed, 2891 skipped, 249 xfailed, 14 xpassed, 476 warnings in 1241.17s (0:20:41) =

osx + blis: 1 failure

=========================== short test summary info ============================
FAILED sparse/linalg/_eigen/lobpcg/tests/test_lobpcg.py::test_failure_to_run_iterations - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
The list of emitted warnings is: [].
= 1 failed, 49579 passed, 2891 skipped, 248 xfailed, 15 xpassed, 475 warnings in 1202.21s (0:20:02) =

rgommers · 2023-01-27T17:34:17Z

Looks great, thanks! The picture gets slightly better with every release cycle it looks like.

We're unlikely to do a 1.9.4 at this point, so let's close this PR and focus on 1.10x?

h-vetinari · 2023-01-28T03:14:25Z

We're unlikely to do a 1.9.4 at this point, so let's close this PR and focus on 1.10x?

I close them as new versions become available. Not planning to invest more time here, but I have a slight preference to leave this open while it's still somewhat relevant (e.g. the last two available versions; will close once we have 1.11).

h-vetinari · 2023-06-26T07:28:04Z

Not that 1.11 is here, let's close this one.

h-vetinari added 6 commits January 27, 2023 18:35

try CI with all blas variants

a88b348

distinguish blis runs for avx512 presence/absence

07536c2

enable setting pagefile size on windows

8aab74c

unskip some tests

f9f9b27

update pypy patch

9e04bfe

MNT: Re-rendered with conda-build 3.23.3, conda-smithy 3.22.1, and co…

09b9ed7

…nda-forge-pinning 2023.01.26.16.59.55

h-vetinari mentioned this pull request Jan 27, 2023

TEST: 1.10.x + blas variants #224

Closed

This was referenced Jan 27, 2023

BUG: test failure of test_lobpcg with certain BLAS versions/flavours scipy/scipy#16927

Closed

BUG: linalg errors on windows + openblas w/ complex numbers scipy/scipy#17869

Closed

h-vetinari mentioned this pull request Jun 26, 2023

TEST: 1.11.x + blas variants #242

Closed

h-vetinari closed this Jun 26, 2023

h-vetinari deleted the 1.9_blas_vars branch June 26, 2023 07:28

h-vetinari mentioned this pull request Jan 23, 2024

TEST: 1.12.x + blas variants #267

Closed

h-vetinari mentioned this pull request Apr 12, 2024

TEST: 1.13.x + blas variants #273

Closed

h-vetinari mentioned this pull request Jun 25, 2024

TEST: 1.14.x + blas variants #280

Closed

h-vetinari mentioned this pull request Jan 8, 2025

TEST: v1.15.x + blas variants #295

Closed

h-vetinari mentioned this pull request Jun 30, 2025

TEST: v1.16.x + blas variants #304

Closed

h-vetinari mentioned this pull request May 27, 2026

TEST: v1.17.x + blas variants #323

Draft

h-vetinari mentioned this pull request Jun 22, 2026

TEST: v1.18.x + blas variants #326

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

TEST: 1.9.x + blas variants#223

TEST: 1.9.x + blas variants#223
h-vetinari wants to merge 6 commits into
conda-forge:v1.9from
h-vetinari:1.9_blas_vars

h-vetinari commented Jan 27, 2023

Uh oh!

conda-forge-webservices Bot commented Jan 27, 2023

Uh oh!

h-vetinari commented Jan 27, 2023 •

edited

Loading

Uh oh!

rgommers commented Jan 27, 2023

Uh oh!

h-vetinari commented Jan 28, 2023

Uh oh!

h-vetinari commented Jun 26, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

h-vetinari commented Jan 27, 2023

Uh oh!

conda-forge-webservices Bot commented Jan 27, 2023

Uh oh!

h-vetinari commented Jan 27, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Scipy 1.9.3

TL;DR: Improvements all around

Notable

The good news:

The bad news

Restrictions

Details

Uh oh!

rgommers commented Jan 27, 2023

Uh oh!

h-vetinari commented Jan 28, 2023

Uh oh!

h-vetinari commented Jun 26, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

h-vetinari commented Jan 27, 2023 •

edited

Loading