Re-render and disable shared Fortran libraries on OSX#69
Re-render and disable shared Fortran libraries on OSX#69jakirkham merged 2 commits intoconda-forge:masterfrom
Conversation
|
Hi! This is the friendly automated conda-forge-linting service. I just wanted to let you know that I linted all conda-recipes in your PR ( |
recipe/meta.yaml
Outdated
| @@ -21,7 +21,7 @@ source: | |||
|
|
|||
| build: | |||
| number: 10 | |||
There was a problem hiding this comment.
Please bump this to 11 as well.
|
Thanks @qwhelan. |
|
Also generally would note this mailing list thread where an HDF5 developer confirms shared libraries are unsupported/don't work properly on Mac with Fortran support enabled. This checks out with their CMake options for Mac as well (see the |
|
Would also appreciate if others took a look at this. |
|
FTR I guess another potential fix to this problem would be to delete the |
Yeah, that sounds more appealing, if it's not a huge pain. Most of my build scripts assume dynamic linking, and don't depend on the Fortran bindings. |
|
Yes, this is still an issue with gnu and intel compilers on the mac. For Intel:
https://software.intel.com/en-us/forums/intel-fortran-compiler-for-linux-and-mac-os-x/topic/543831?language=de
and I’ve tested with gcc 6 and the fortran tests still don’t pass. I have yet to report the bug to gnu.
Scot
On May 16, 2017, at 8:20 AM, Stuart Berg <notifications@github.com<mailto:notifications@github.com>> wrote:
I guess another potential fix to this problem would be to delete the dylib Fortran files produced by the HDF5 build on Mac
Yeah, that sounds more appealing, if it's not a huge pain. Most of my build scripts assume dynamic linking, and don't depend on the Fortran bindings.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#69 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AIO1hyi9wbUtXxAxDFUIbxbJv8Jbr18Yks5r6aK0gaJpZM4Nb_Qp>.
|
|
Do you know of a way to selectively disable shared libraries built, @brtnfld? Would be nice if we could keep all shared libraries except the Fortran ones if they are the issue. |
|
One thing I noticed is that the make files will have to be patched to not execute the Fortran test suite after the dynamic Fortran libs are built. Also, I do not have access to a Mac so some of these modifications may be difficult for me to experiment with. |
|
Sorry, I don't follow. FWICT the failing tests occur during |
|
@jakirkham Whether Fortran libs should be built is passed to / stored by |
|
If configure detects enable-fortran and the OS is Darwin, then it will not build fortran shared libraries. This is what is built on my mac (no fortran shared libraries):
drwxr-xr-x 6 brtnfld 204 May 17 08:46 ../
-rwxr-xr-x 1 brtnfld 932 May 17 08:46 libdynlib1.la<http://libdynlib1.la>*
-rwxr-xr-x 1 brtnfld 8448 May 17 08:46 libdynlib1.so*
-rwxr-xr-x 1 brtnfld 932 May 17 08:46 libdynlib2.la<http://libdynlib2.la>*
-rwxr-xr-x 1 brtnfld 8448 May 17 08:46 libdynlib2.so*
-rwxr-xr-x 1 brtnfld 932 May 17 08:46 libdynlib3.la<http://libdynlib3.la>*
-rwxr-xr-x 1 brtnfld 8672 May 17 08:46 libdynlib3.so*
-rwxr-xr-x 1 brtnfld 932 May 17 08:46 libdynlib4.la<http://libdynlib4.la>*
-rwxr-xr-x 1 brtnfld 8944 May 17 08:46 libdynlib4.so*
-rwxr-xr-x 1 brtnfld 942 May 17 08:46 libdynlibadd.la<http://libdynlibadd.la>*
-rwxr-xr-x 1 brtnfld 8448 May 17 08:46 libdynlibadd.so*
-rwxr-xr-x 1 brtnfld 947 May 17 08:46 libdynlibdiff.la<http://libdynlibdiff.la>*
-rwxr-xr-x 1 brtnfld 8456 May 17 08:46 libdynlibdiff.so*
-rwxr-xr-x 1 brtnfld 947 May 17 08:46 libdynlibdump.la<http://libdynlibdump.la>*
-rwxr-xr-x 1 brtnfld 8456 May 17 08:46 libdynlibdump.so*
-rwxr-xr-x 1 brtnfld 937 May 17 08:46 libdynlibls.la<http://libdynlibls.la>*
-rwxr-xr-x 1 brtnfld 8456 May 17 08:46 libdynlibls.so*
-rwxr-xr-x 1 brtnfld 947 May 17 08:46 libdynlibvers.la<http://libdynlibvers.la>*
-rwxr-xr-x 1 brtnfld 8944 May 17 08:46 libdynlibvers.so*
-rwxr-xr-x 1 brtnfld 7948208 May 17 08:46 libhdf5.1000.dylib*
-rw-r--r-- 1 brtnfld 11476672 May 17 08:46 libhdf5.a
lrwxr-xr-x 1 brtnfld 18 May 17 08:46 libhdf5.dylib@ -> libhdf5.1000.dylib
-rwxr-xr-x 1 brtnfld 960 May 17 08:46 libhdf5.la<http://libhdf5.la>*
-rw-r--r-- 1 brtnfld 2915 May 17 08:46 libhdf5.settings
-rw-r--r-- 1 brtnfld 998112 May 17 08:46 libhdf5_fortran.a
-rwxr-xr-x 1 brtnfld 989 May 17 08:46 libhdf5_fortran.la*
-rwxr-xr-x 1 brtnfld 161928 May 17 08:46 libhdf5_hl.1000.dylib*
-rw-r--r-- 1 brtnfld 205448 May 17 08:46 libhdf5_hl.a
lrwxr-xr-x 1 brtnfld 21 May 17 08:46 libhdf5_hl.dylib@ -> libhdf5_hl.1000.dylib
-rwxr-xr-x 1 brtnfld 1033 May 17 08:46 libhdf5_hl.la*
-rw-r--r-- 1 brtnfld 675416 May 17 08:46 libhdf5hl_fortran.a
-rwxr-xr-x 1 brtnfld 1116 May 17 08:46 libhdf5hl_fortran.la*
what HDF5 version are you trying to build?
On May 17, 2017, at 12:23 AM, jakirkham <notifications@github.com<mailto:notifications@github.com>> wrote:
Do you know of a way to selectively disable shared libraries built, @brtnfld<https://github.com/brtnfld>? Would be nice if we could keep all shared libraries except the Fortran ones if they are the issue.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#69 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AIO1h7LY47qgmKJoq7isEsQqNPGJxoyQks5r6oRigaJpZM4Nb_Qp>.
|
| ] %} | ||
| {% for each_hdf5_lib in hdf5_libs %} | ||
| - test -f $PREFIX/lib/lib{{ each_hdf5_lib }}.a # [unix] | ||
| - test -f $PREFIX/lib/lib{{ each_hdf5_lib }}.dylib # [osx] |
There was a problem hiding this comment.
Could you please add this back?
There was a problem hiding this comment.
@jakirkham I can, but it will fail when executed unless we move the fortran lib into a different list. I'll revert this when I make the delete-between-make-and-make-check patch.
There was a problem hiding this comment.
Yep, we can break out the Fortran case no problem. Ok SGTM.
recipe/meta.yaml
Outdated
| - test -f $PREFIX/lib/lib{{ each_hdf5_lib }}.dylib # [osx] | ||
| - test -f $PREFIX/lib/lib{{ each_hdf5_lib }}.so # [linux] | ||
| # Commented out as shared libs not currently supported for OSX | ||
| # - test -f $PREFIX/lib/lib{{ each_hdf5_lib }}.dylib # [osx] |
recipe/meta.yaml
Outdated
| {% endfor %} | ||
| # {% for each_hdf5_lib in hdf5_libs %} | ||
| # - otool -L $PREFIX/lib/lib{{ each_hdf5_lib }}.dylib # [osx] | ||
| # {% endfor %} |
recipe/meta.yaml
Outdated
| - if not exist %PREFIX%\\Library\\bin\\{{ each_hdf5_lib }}.dll exit 1 # [win] | ||
| {% endfor %} | ||
|
|
||
| # Commented out as shared libs not currently supported for OSX |
So I think @qwhelan has already explained this quite nicely above, but it seems |
|
I did not realize you are enabling unsupported, if you don’t mind me asking, why are you specifying unsupported? I will have to check on whether to always disabling shared fortran even when unsupported is specified. We currently use unsupported to check if newer compilers have fixed the issue.
On May 17, 2017, at 9:18 AM, jakirkham <notifications@github.com<mailto:notifications@github.com>> wrote:
If configure detects enable-fortran and the OS is Darwin, then it will not build fortran shared libraries. This is what is built on my mac (no fortran shared libraries).
So I think @qwhelan<https://github.com/qwhelan> has already explained this quite nicely above, but it seems --enable-unsupported is overriding this, which seems like a bad idea. In particular, the lines below in this build log<https://s3.amazonaws.com/archive.travis-ci.org/jobs/216319923/log.txt> tell us HDF5 is doing this.
checking if shared Fortran libraries are supported... no
configure: WARNING: Shared Fortran libraries not currently supported on Mac.
configure: WARNING: Allowing unsupported Fortran shared libraries due to use of --enable-unsupported flag
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#69 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AIO1h6mTeCjwb76HGZrBuBooqnDB5n1zks5r6wG1gaJpZM4Nb_Qp>.
|
|
Some contributors wanted to have |
|
We will try to resolve the issue in the next release of 1.8 and 1.10 to always disable shared on mac with fortran, though it’s now a little more complicated because Intel fixed their bug, so HDF5 works with intel fortran 17 and shared libraries on the mac.
Scot
On May 17, 2017, at 10:15 AM, jakirkham <notifications@github.com<mailto:notifications@github.com>> wrote:
Some contributors wanted to have --enable-threadsafe. However as we build the Fortran and C++ interfaces, that had to come with --enable-unsupported to satisfy existing use cases.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#69 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AIO1h0AQySMBHuwQr-aqq8JkhEoMQMfjks5r6w75gaJpZM4Nb_Qp>.
|
|
If you need any help resolving this issue, @bluescarni might be able to help... |
|
@jakirkham Ended up reading through the This builds the static version of the Fortran API on OSX (so |
recipe/meta.yaml
Outdated
| number: 10 | ||
| skip: True # [py==36] | ||
| number: 11 | ||
| skip: True # [win and py==36] |
There was a problem hiding this comment.
Minor point, but should be able to change py==36 to py36.
There was a problem hiding this comment.
Will give that a shot - left it as is due to py36 not being listed on https://conda.io/docs/building/meta-yaml.html
There was a problem hiding this comment.
Hmm...should be added. Raised issue ( conda/conda-docs#443 ) to get py36 added.
appveyor.yml
Outdated
|
|
||
| - TARGET_ARCH: x64 | ||
| CONDA_PY: 36 | ||
| CONDA_INSTALL_LOCN: C:\\Miniconda36-x64 |
There was a problem hiding this comment.
Not sure why this is here as the skip should have ensure it wasn't added. Could you please try another re-render to see if that removes this? If not, we will need to investigate.
There was a problem hiding this comment.
Good catch, re-rendering removed this
Nice find! Thanks for doing that @qwhelan. Maybe if we could set this with an environment variable that would solve this issue cleanly, @brtnfld. Would point out to others that one can see shared Fortran libraries are disabled.
SGTM. |
|
@jakirkham Incorporated all of your comments with this last push, should be good to merge once tests finish. |
jakirkham
changed the title
jakirkham
changed the title
|
Might be worth updating the PR description since we ended up not using the flag, but using a patch to disable shared Fortran libraries. |
|
@jakirkham Done. |
|
It's in. Thanks @qwhelan. |
5 participants
While digging into why the Fortran tests were failing on OSX, I came across a log message saying the following:
The
--enable-unsupportedflag is needed to build the high-level API and enable thread-safe (which is not supported by the library authors), so we can't turn that off. The solution here is to patchconfigureto disable the building of shared libraries for the Fortran API only.Additionally, I have re-rendered the feedstock.
cc @jakirkham