Scikit buildsystem #38

repagh · 2018-08-21T12:32:39Z

After many iterations, a better set of changes for using scikit-build.
Works with conda build and with direct python/pip builds

This still contains a workaround for the lack of recent (>= 0.6.1) scikit-build on conda-forge. Directly installs & uninstall scikit-build using pip as part of a conda build. Can be simplified later.

Recent scikit build (0.8.1) available both on conda-forge and pip correctly build this.

…into scikit-buildsystem

moorepants · 2018-08-21T17:35:58Z

FYI, this will hopefully be merged today: conda-forge/scikit-build-feedstock#11

moorepants · 2018-08-22T12:16:22Z

FYI, conda forge now has scikit-build 0.8.0.: conda-forge/scikit-build-feedstock#14

repagh · 2018-08-22T12:18:27Z

Yes, works. Am using the conda-forge scikit-build now, with a conda install for CONDA=0, and through the meta.yaml for CONDA=1

repagh · 2018-09-05T07:34:45Z

@murrayrm , @moorepants, can this be merged? It uses the scikit-build currently in conda-forge (no longer needs the hack), builds and runs successfully on Linux, OSX, and Windows.

roryyorke · 2018-10-09T18:43:13Z

I can't get this to build on Ubuntu 18.04 64-bit. Conda version 4.5.11, conda build 3.16.0. Build command conda build -c conda-forge --python=3.6 slycot/conda-recipe-openblas (from directory containg slycot). The error I get is lengthy, but I think the important bit is

  File "/home/rory/.miniconda3/conda-bld/slycot_1539023903021/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.6/site-packages/skbuild/command/bdist_wheel.py", line 6, in <module>
    from wheel import archive as _wheel_archive
ImportError: cannot import name 'archive'

I've attached a complete build log. Any ideas what I'm doing wrong?
build-3.6.log

moorepants · 2018-10-09T19:17:11Z

@murrayrm , @moorepants, can this be merged? It uses the scikit-build currently in conda-forge (no longer needs the hack), builds and runs successfully on Linux, OSX, and Windows.

LGTM

moorepants · 2018-10-09T19:19:09Z

Actually I have one question. Since setup.py has changed and now requires scikit build, this needs to be made an explicit build dependency of the package. And will it work without using conda and conda-forge?

murrayrm · 2018-10-09T19:23:48Z

I'd also like to test this on MacOS and make sure that it still works. Has anyone tried it?

roryyorke · 2018-10-13T16:07:45Z

And today it builds. conda-build 3.16.1, and scikit-build 0.8.1 (previously 0.8.0). I've got the build log if anyone's interested.

moorepants · 2018-10-13T17:16:47Z

setup.py

@@ -5,7 +5,7 @@
 Slycot wraps the SLICOT library which is used for control and systems analysis.

 """
-from __future__ import division, print_function
+from skbuild import setup


The installation documentation will need to let users know of this new requirement, as well as cmake (and any other build tools that now need to be installed). I personally think that skbuild should be an optional requirement because a user can build it without this if they so desire.

I added information on scikit-build and cmake requirements to the readme file.

I don't think it is feasible to maintain two buildsystems. I got to use cmake/skbuild after repeatedly trying to use and tweak the numpy distutils, however I found no way to specify flang to the distutils buildsystem, while flang works wonderfully for Windows builds.

The buildsystem does not rely on conda; if skbuild and cmake are available, local builds work fine.

Ok, seems reasonable.

…E.rst

roryyorke · 2018-12-28T15:05:11Z

this had a failure on Dec. 8 https://travis-ci.org/python-control/Slycot/jobs/465308309 ; I reran one case (Python 3.6, TEST_CONDA=1) today without changing anything, and it passed https://travis-ci.org/roryyorke/Slycot/builds/473027170. Will try a rebuild (feels a bit pointless, but what the heck...).

According to [1], use Miniconda2-latest-Linux-x86_64.sh. This fixes a build bug on Travis. [1] https://conda.io/docs/user-guide/tasks/use-conda-with-travis-ci.html, retrieved 2018-12-28.

conda-recipe-openblas/bld.bat

Co-Authored-By: repagh <[email protected]>

…into scikit-buildsystem

repagh · 2019-01-07T14:40:03Z

There seems to be something wrong with the conda-build for python 2.7. Have not found a fix for that yet.

roryyorke · 2019-01-08T15:29:22Z

Doesn't repagh/Slycot#2 fix the Python 2 build? See https://travis-ci.org/roryyorke/Slycot/builds/473091198

Update Miniconda URL from Conda docs.

repagh · 2019-01-10T10:27:07Z

Doesn't repagh#2 fix the Python 2 build? See https://travis-ci.org/roryyorke/Slycot/builds/473091198

Good catch. It now works!

roryyorke · 2019-01-10T18:32:14Z

If there are no objections to this in the next week, I propose merging to master.

In favour:

possible to build on Windows; hopefully we can add Windows packages to conda-forge (@moorepants, is this practical?)
better build output; we don't have that 1000 (or whatever) character link command appearing in the build anymore (this is fairly minor)

Two downsides:

scikit-built doesn't support building wheels
scikit-build also doesn't support "setup.py develop" mode

moorepants · 2019-01-10T22:22:20Z

is this practical?

I am not quite sure. As far as I know things that rely on Fortran are not possible to compile on Windows with the conda forge system. But maybe things have changed in the last months. The last I could understand was that flang might be the only way forward there.

moorepants · 2019-01-10T22:27:21Z

Looks like numpy is built for all OSs on conda forge, but scipy isn't build on windows for some reason. It maybe be that there is still some hurdle.

moorepants · 2019-01-10T22:32:56Z

It looks like the numpy feedstock installs from the numpy wheel on pypi (which is build with ATLAS on windows) and doesn't actually build on appveyor. The best bet would be to build slycot wheels for windows manually and then have the conda forge recipe install them.

ilayn · 2019-01-11T08:12:28Z

Since v.1.0 we are able to build SciPy on Windows via OpenBLAS. It is still definitely not a walk in the park to build the BLAS of choice but if you just use a prebuilt BLAS flavor it is quite possible (I will modify the notes with a correction soon scipy/scipy#9669).

moorepants · 2019-01-11T22:59:00Z

It'd be great to get that working on conda forge so that other packages that need to build on windows would have a working example.

repagh · 2019-01-14T10:06:42Z

I use a pretty bare Windows (in a virtualbox) to build the conda packages with scikit, with the following manually installed:

Installation of msvc 2015 for the c parts
miniconda3
git, but only for getting the software

For the build this uses the conda-forge flang (available win64 only), cmake, scikit-build, etc. I don't know what the conda-forge windows buildsystem looks like, but I would guess this would be compatible.

p.s. python setup.py bdist_wheel simply works. I tried that against the openblas from conda-forge, you would need to have a wheel-based openblas to use it outside the conda ecosystem.

moorepants · 2019-01-14T14:38:36Z

As soon as this is merged we can give it a shot on conda-forge and see what happens.

roryyorke · 2019-01-15T04:42:47Z

Sorry, I thought I saw an issue about scikit-build and wheels, but the only one I can find now is closed.

I may be wrong about "setup.py develop" too; see, e.g., scikit-build/scikit-build#371

repagh added 3 commits August 21, 2018 14:17

Using scikit-build as buildsystem

8c528a3

Using scikit-build as buildsystem

b3168f4

Merge branch 'scikit-buildsystem' of https://github.com/repagh/Slycot …

e6af8a7

…into scikit-buildsystem

repagh mentioned this pull request Aug 21, 2018

Build with scikit build #33

Closed

repagh added 3 commits August 21, 2018 15:47

build command modifications for .travis.yml

aba817d

- For travis, do not re-build through runtest.py

3d12649

- do not run tests from Slycot dir

adc13d7

repagh added 2 commits August 22, 2018 13:49

- since scikit-build is now in conda-forge, could remove the hack

9d435d4

- explicitly selecting CMAKE_GENERATOR="Unix Makefiles"

13b3fe6

repagh mentioned this pull request Aug 24, 2018

Problem with Slycot installation on Ubuntu #15

Closed

moorepants reviewed Oct 13, 2018

View reviewed changes

Added information on scikit-build and cmake build dependency on READM…

2313855

…E.rst

Update Miniconda URL from Conda docs.

a5d0d78

According to [1], use Miniconda2-latest-Linux-x86_64.sh. This fixes a build bug on Travis. [1] https://conda.io/docs/user-guide/tasks/use-conda-with-travis-ci.html, retrieved 2018-12-28.

roryyorke reviewed Dec 29, 2018

View reviewed changes

conda-recipe-openblas/bld.bat Outdated Show resolved Hide resolved

roryyorke mentioned this pull request Dec 30, 2018

Don't use deprecated np.matrix in tests. #43

Merged

roryyorke and others added 3 commits January 7, 2019 12:43

Update conda-recipe-openblas/bld.bat

75d6867

Co-Authored-By: repagh <[email protected]>

RoryYorke's build fix on .travis.yml

1d686b7

Merge branch 'scikit-buildsystem' of https://github.com/repagh/Slycot …

32d85a1

…into scikit-buildsystem

Merge pull request #2 from roryyorke/rory/scikit-buildsystem-travis-py27

c05a78f

Update Miniconda URL from Conda docs.

roryyorke merged commit b82299b into python-control:master Jan 18, 2019

roryyorke mentioned this pull request May 1, 2019

Bringing out release 0.3.4 #49

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Scikit buildsystem #38

Scikit buildsystem #38

repagh commented Aug 21, 2018 •

edited

Loading

moorepants commented Aug 21, 2018

moorepants commented Aug 22, 2018

repagh commented Aug 22, 2018

repagh commented Sep 5, 2018

roryyorke commented Oct 9, 2018

moorepants commented Oct 9, 2018

moorepants commented Oct 9, 2018

murrayrm commented Oct 9, 2018

roryyorke commented Oct 13, 2018

moorepants Oct 13, 2018

repagh Dec 8, 2018

moorepants Dec 8, 2018

roryyorke commented Dec 28, 2018

repagh commented Jan 7, 2019

roryyorke commented Jan 8, 2019

repagh commented Jan 10, 2019

roryyorke commented Jan 10, 2019

moorepants commented Jan 10, 2019

moorepants commented Jan 10, 2019

moorepants commented Jan 10, 2019

ilayn commented Jan 11, 2019

moorepants commented Jan 11, 2019

repagh commented Jan 14, 2019 •

edited

Loading

moorepants commented Jan 14, 2019

roryyorke commented Jan 15, 2019

Scikit buildsystem #38

Scikit buildsystem #38

Conversation

repagh commented Aug 21, 2018 • edited Loading

moorepants commented Aug 21, 2018

moorepants commented Aug 22, 2018

repagh commented Aug 22, 2018

repagh commented Sep 5, 2018

roryyorke commented Oct 9, 2018

moorepants commented Oct 9, 2018

moorepants commented Oct 9, 2018

murrayrm commented Oct 9, 2018

roryyorke commented Oct 13, 2018

moorepants Oct 13, 2018

Choose a reason for hiding this comment

repagh Dec 8, 2018

Choose a reason for hiding this comment

moorepants Dec 8, 2018

Choose a reason for hiding this comment

roryyorke commented Dec 28, 2018

repagh commented Jan 7, 2019

roryyorke commented Jan 8, 2019

repagh commented Jan 10, 2019

roryyorke commented Jan 10, 2019

moorepants commented Jan 10, 2019

moorepants commented Jan 10, 2019

moorepants commented Jan 10, 2019

ilayn commented Jan 11, 2019

moorepants commented Jan 11, 2019

repagh commented Jan 14, 2019 • edited Loading

moorepants commented Jan 14, 2019

roryyorke commented Jan 15, 2019

repagh commented Aug 21, 2018 •

edited

Loading

repagh commented Jan 14, 2019 •

edited

Loading