TEST: 1.19.x + blas variants#196
Conversation
|
Hi! This is the friendly automated conda-forge-linting service. I just wanted to let you know that I linted all conda-recipes in your PR ( |
|
This looks quite a bit better than on the scipy-side. Still it did catch some errors that might not show up anywhere else in CI. I mentioned in conda/conda-build#3947 that IMO, foundational packages like numpy/scipy should be tested across all available blas variants. This would - effectively - provide some further CI on the various blas builds, but also make sure that no broken variants are shipped. Note: The cells of the matrix have the same behaviour for all python versions (cpython 3.6-3.8, pypy 3.6) unless otherwise noted - hence the simplified matrix
There were 4 additional failures not shown in the matrix above:
linux + x86 + netlib/openblas + pypy: segfault, test suite abortedlinux + ppc64le + openblas + cpython 3.6: 1 test failure (TestCholesky.test_basic_property)linux + ppc64le + openblas + cpython 3.7: 4 test failureslinux + ppc64le + openblas + cpython 3.8: 3 test failures |
|
Maybe worth pinging @conda-forge/help-pypy on this. The segfaults on linux+pypy for numpy 1.19 affect both this PR and #195. |
|
What version of OpenBLAS are you testing with? 0.3.10 was released recently and fixes an interaction that would cause a segfault on PyPy on azure machines. |
|
@mattip There's more stuff that I don't understand, actually. Bumping blis from 0.6.1 to 0.7.0 has introduced a segfault on cpython 3.6, 12 test failures on 3.7, but runs through fine on 3.8...? 🤔 |
|
After bumping blis to 0.7.0 and openblas to 0.3.10 (see conda-forge/blas-feedstock#45), the picture changed a bit, in some ways unexpectedly. In short:
linux + x86 + mkl + pypy: segfault, test suite abortedlinux + ppc64le + openblas + cpython: 3 test failureslinux + ppc64le + openblas + pypy: 4 test failureswindows + blis + cpython: 11-12 failed testsThis one seems to be a bit fickle as well - in the previous run, it appeared for py36 & py37 but not py38, not it appeared for py37 & py38 but not py36. |
|
Thanks for the summary. Numpy regularly builds on travisCI machines with PyPy, both in the numpy/numpy CI and on the MacPython/num-wheels CI. I wonder what is different on the Azure builds? |
|
In the build log for pypy I am seeing |
You're looking at the right logs, but I think you're mixing up the reference-lapack version numbers with openblas (note: 3.9.0 isn't in conda-forge yet because MKL isn't compatible yet). As I noted in the last comment, the segfault on pypy seems non-deterministic. It appeared for openblas in 2 out of 3 runs, for netlib 1/3 times and for mkl 1/3 times. One of the runs has been deleted by azure, but here's one where the segfault occurred with openblas. I haven't investigated which test follows |
|
Ahh, right sorry. But in the In NumPy we use add a check like this (from |
You're right, sorry. The run I linked was before conda-forge/blas-feedstock#45 was merged. The first run after that had the segfault, but got deleted by azure since I restarted before everything was through. Let's try that again. |
|
this is a tough PR. The starts must align just perfectly for everything to pass in the same run. |
It seems to me the best answer would be to figure out what causes the segfault and fix it 😉 The blis+win builds are also failing consistently, as are the linux+ppc builds. So we'll probably need more than lucky stars to pull this into the green 😅 |
|
I wonder if there is a path to release a 1.19 NumPy for at least the most common platforms. |
|
I see that OpenBLAS is being built with DetailsIn [this comment](https://github.com/OpenMathLib/OpenBLAS/issues/2388#issuecomment-582267407), and elsewhere, they recommends using `TARGET` as a minimal platform when `DYNAMIC_ARCH` is used which will be the fallback when no better dynamic loop can be found. The default is the build platform, which may be problematic if the build is done on a more advanced machine than the test. |
…nda-forge-pinning 2022.04.12.21.56.28
This reverts commit 6dc144f.
b1d98ba to
f5d4c8b
Compare
Revival (new PyPy builds and BLAS updates): all green except PPCDue to rebuilding 1.19 for pypy3.8/3.9, much less several relevant BLAS (& infrastructure) changes, I thought I'd revive this PR for one last update. From 9 failures out of 92 runs, we're now at 12 failures (PPC-only) out of 108 runs. Notable
Details
Build logs: |
Based on #195; doing a similar investigation as in conda-forge/scipy-feedstock#130.